jfflLGARD and MARQUIS’ 

■nditioning and 
learning 


revised by 

Gregory A. Kimble 

duke university 


Second Edition 


New 



York 


appleton-century-cuoits. INC. 



Copyright (c) 1961 by 
APPLETON-CENTUEY-CROETS, INC. 


All rights reserved. This tool:, or parts 
thereof, must not he reproduced in any 
form without permission of the publisher. 

611-8 

Library of Congress Card Number: 61-5440 


Copyright © 1940 by D. Appleton 


-Century Company, 


PP.IKTED IK THE 


XTKITED STATES 


0F AMERICA 


Inc. 



PREFACE 


Since the publication of the first edition of this book much has hap- 
pened in the fields of conditioning and learning. New theories have ap- 
peared and old ones have been given up or modified. There have been 
important advances in experimental design and statistics. Knowledge of 
the philosophy of science, considered esoteric in the 1930’s, is now com- 
monplace. Experimental technology has seen a vast improvement. And 
the amount of research which has been done is enormous. 

Naturally these developments have had an influence upon this revision. 
Most important has been a restriction of the range of materials covered. 
A comparison of tables of contents will show that four of the original 
chapters have been omitted — those on serial learning, problem solving, 
voluntary action, and neurophysiological mechanisms. It has been pos- 
sible to include some of the content of these chapters elsewhere in the 
revision ; but with the exception of the physiological materials, the amount 
of space devoted to these topics is significantly reduced. In place of the 
four omitted chapters there are four new ones. Two of these resulted 
from the subdivision of the original chapters on the nature of reinforce- 
ment and on generalization and discrimination. The two other additional 
chapters, on secondary reinforcement and motivation, are almost totally 
new to this edition. 


In spite of these, and other, modifications, this revision retains the 
essential characteristic of the parent edition, that of being basically em- 
pirical rather than theoretical in approach. To paraphrase the statement 
of Hilgard and Marquis, in their preface written in 1 910: This book 
makes no effort to present a finished theory of learning. The attempt, 
instead has been to place the facts and alternative conceptions of the 
nature of learning in some sort of order. It may be that such ordering 
will prove to be a small step in the direction of the formulation of a 
better theory than is now available. 


There are many people to whom I owe a considerable debt of gratitude 
for help in the preparation of this revision. In the first place, there are 
fte hundreds of individuals who generously contributed repents, an- 



PREFACE 


vi 

published manuscripts, and, frequently, sound advice on particular 
topics. Tlien there are those who assisted in the tedious details: Mrs. 
Margaret S. King, who did most of the job of assembling a bibliography 
of some 3,000 titles, half of which found their way into the book ; Mrs. 
Ann Boneau, who served as copy editor, as well as typist; my wife, 
Lucille L. Kimble, who prepared most of the figures ; and my two children, 
Jeffrey and Judith, who kept my card file more or less in order. I am 
especially grateful to R. M. Elliott, Kenneth MacCorquodale, and Harold 
Schlosberg, who read much or all of the manuscript and made suggestions 
which were invaluable. Finally, I should like to thank the following 
publishers for permission to reproduce figures or quotations : The Ameri- 
can Psychological Association, for its various journals, The American 
Journal of Psijcliology, Appleton-Century-Crofts, Cambridge University 
Press, Hareourt, Brace, Holt, Rinehart and Winston, The Humanities 
Press, McGraw-Hill, The New York Academy of Sciences, Prentice-Hall 
The Psychological Record The Ronald Press, Scientific American Uni- 
versity of California Press, Williams and Wilkins, and Yale University 
Press. In each case, acknowledgment to the original source is given with 
the material reproduced. 3 


G. A. K. 



preface to the first edition 


Theories of learning which make use of conditioning principles are 
not related in a simple manner to the facts from conditioning experi- 
ments. This book represents an effort to place the facts and theories into 
some sort of order through critical exposition. The net result is not a 
finished theory ; in man y instances we have found it necessary to point 
to several alternative conceptions. The examination of the relation of 
conditioning to other basic learning experiments has at other points re- 
vealed serious gaps in our knowledge which future experimentation must 
fill. 

Although conditioning has sometimes been proposed as a basic concept 
for the fields of cerebral physiology, of mental hygiene and personality, 
and of the higher thought processes, we have found it desirable to place 
greatest emphasis upon the relation of conditioning to learning theory. 
In the later chapters we have surveyed the possibilities of applying con- 
ditioning concepts within some of the other fields. 

Our own experimental work in conditioning was first undertaken in 
the laboratories of Professors Raymond Dodge, Harold S. Burr, and John 
F. Fulton of Yale University, and to these and other teachers, colleagues, 
and collaborators, we owe much. We have profited greatly from the read- 
ing and criticism of the entire manuscript by Professor R. M. Elliott, 
editor of this series. The manuscript was improved through suggestions 
following critical reading of different portions by R. L. French, V. E. 
Hall, G. L. Heathers, W. T. Heron, C. I. Hovlnnd, C. L. Hull, J. L. 
Kennedy, A. A. Lumsdaine, T. L. McCulloch, N. E. Miller, Helen Peak, 

T. C. Ruch, R. R. Sears, B. F. Skinner, K. W. Spence, E. C. Tolman, and 
S. B, Williams. That their suggestions were all useful to us does not mean 
that they have approved the use which we made of them. The kindness 
of Dr. G. H. S. Razran in furnishing us an advance copy of his bibliog- 
raphy is gratefully acknowledged. lie and several other authors kindly 
placed in our hands manuscripts prior to publication. Material has been 
drawn upon from personal communications received from the following; 

W. H. Burnham, W. II. Gantt, K. S. Lash Icy, R. B. houcks, Florence 
Mateer, J. J. B. Morgan, Richard Parmcnter, J. B. Watson, and It. 31. 
Yerkes. 


rii 
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Acknowledgment is made to D. Appleton-Century Co., The Macmillan 
Company, Oxford University Press, and Prentice-Hall, Inc., from whose 
publications figures or quotations are reproduced, and to the editors and 
publishers of the following journals for similar reproductions : American 
Journal of Psychology, Archives of Psychology, Journal of Comparative 
Psychology, Journal of Experimental Psychology, Journal of General 
Psychology, Journal of Genetic Psychology, Proceedings of the National 
Academy of Sciences, Psychological Monographs, Psychological Review, 
and Psychosomatic Medicine. Citation to the original source is given in 
each case. 

E. R. H. 

D. G. M. 
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The Definition of Learning 

The term conditioning as it is used in present-day psychology refers to 
two procedures, known as classical conditioning and instrumental condi- 
tioning. The first of these procedures derives from the early work of 
Pavlov; the latter stems from that of Bechterev and Thorndike. Together 
these two simple experiments, with their superstructures of theory, have 
had a profound effect upon the history of psychology and its related " 
sciences. 

The classical conditioning experiment is familiar to every student of 
physiology and psychology. When meat powder is placed in a dog's 
mouth, saliva flows in response to this unconditioned stimulus (tJCS). 
This natural and relatively automatic behavior is called an unconditioned 
reflex or unconditioned response (UCR). An incidental stimulus, such as 
the sound of a ringing bell, does not at first result in the flow of saliva. 

If the bell sounds each time just before the food is presented, however, it 
soon comes to elicit salivation. The new response to the bell is called a 
conditioned reflex or conditioned response (CR), and the bell is called n 
conditioned stimulus (CS). A relationship (association) has boon estab- 
lished between the bell and food, so that a response resembling the one 
originally made to the food is now made to the bell. 

The instrumental conditioning experiment, perhaps less well-known 
outside of psychology, has been more popular within the field. In a typical 
experiment, a hungry rat is placed in a soundproof compartment into 
which a small bar or other manipulandum can be inserted. Whenever the 
animal presses the bar, food, in the form of a small pellet, is delivered 
into a tray for the rat to seize and consume. With practice, the respond , 
of pressing the bar, which the rat first discovered in exploring the ap- 
paratus, becomes the most salient feature of the rat 's behavior. So long 
as the animnl remains hungry and receives food for pressing the bar be 
concentrates on this response. 

These two experimental procedures differ in important methodological 
ways and may also differ in terms of the principles which underlie them. 

For our present purposes, it is important only to note that they are rela- 
tively simple forms of learning. 
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The term learning is broader than tbe term conditioning , since it cover, 
a range of processes from tbe classical conditioned reflex to human prob- 
lem solving Some writers believe that conditioning is a prototypical form 
Si This may or may not be true. Different views of the re ation- 
ship between conditioning and learning are treated m Chapter 2. Chap er 
1 discusses the more general concept in order to introduce a number o 
important ideas which reappear at later points m the booh. Among these 
are the most important theoretical issues in the field of learning and the 
most fundamental questions of fact. 


Factual Definitions 

Definitions of learning are of two different general sorts which we will 
refer to as factual and theoretical definitions. The factual definitions are 
all alike in that they relate the phenomenon of learning to observable 
events in the physical world. The theoretical definitions describe the es- 
sential conditions or the basic processes which the writer believes to be 
necessary for learning to occur. 

As regards the factual definitions, there has always been general agree- 
ment among authorities on the subject that learning refers to a more or 
\less permanent change in behavior which occurs as a result of practice. 
In such a statement, both the dependent variable (changes in behavior) 
and the independent variable (practice) are reasonably objective. The 
term learning itself has the status of an intervening, unobserved variable 
linking these two sets of observables. Several expressions of this general 
point of view selected from the writings of psychologists of very different 
theoretical opinions are the following : 


tiVo consider any systematic change in behavior to he learning whether or not the 
change is adaptive, desirable for certain purposes, or in accordance with any 
other such criteria. (Bush and Mosteller, 1955) 

These changes in behavior which follow behavior we shall call learning. (Guthrie, 
1952) 


. . . we may say that learning is taking place whenever behavior shows a pro_ 
gressive change or trend with a repetition of the same stimulating situation and 
when the change cannot be accounted for on the basis of fatigue or of receptor 
and effector changes. (W. S. Hunter, 1934) 


Learning, as we measure it, is a change in performance which 
ditions of practice. (McGeocli and Irion, 1952) 


occurs under con- 


Lcarning is a more or less permanent incremental modification of behavior which 
rc alts from activity, special training, or observation. (Munn, 1955) 

Vc can define learning as that process which manifests itself by adaptive changes 
in individual behavior as a result of experience. (Thorpe, 1956) 
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Any more than transient modification o£ behavior which presumably results from 
a " 4 not fr °“ change. (Wenger, Jones, and Jones, 


Snch definitions accomplish several things. First, by limiting learnin- 
to relatively permanent changes, they exclude modifications of behavior 
due to motivational factors, to sensory adaptation, or to fatigue. Second 
by identifying practice, training, or experience as the essential condition 
for learning, they exclude behavioral changes resulting from maturation, 
senescence, or physiological variables. Third, by implication they establish 
the status of the concept learning as an intervening variable (Bergmann 
and Spence, 1941, 1944; MacCorquodale and Meebl, 1948). The diagram 
below presents the essentials of the various definitions in a iray ivhich 
makes this last point. 


Suggested independent variables 


Repetition 

Activity 

Behavior 

Practice > 

Training 

Observation 

Experience I 


Learning 


Suggested dependent variables 


Trend in behavior 
Changes in behavior 
Incremental modification 
of behavior 


The right-hand side of this schema contains a series of obviously par- 
allel alternatives expressing the idea that learning produces progressive 
changes in behavior. The left-hand terms are all expressions to describe 
the successive exposure of the organism to the same situation. Even the 
term experience seems to be meant in this sense by those who use it in 
such definitions. Mnier and Sclmeirla, who are most explicit on this point, 
say that “the term experience as used here . . . refers to the effect of 
stimulation on the organism” (1935, p. 337), a statement which makes 
experience ns objective as practice or training. At the same time, the par- 
ticular expressions used in a factual definition arc often predictive of the 
theoretical view espoused by a given author. The connotations of the term 
experience, for example, are such as to leave no one surprised when, later 
on, Maier and Sehneirla adopt a cognitive theory for certain forms of 
learning. This Inst observation should not be permitted, however, to ob- 
scure the fact that, at the factual level, there is good ncrocmcnt as to the 
meaning of the term learning. 

Another point worth making with respect to the factual definitions of 
learning is that they are operational definitions (Pratt, 19 IS) in that 
they reduce the concept to observations which can be made on the pby«i- 
ca! world. As we have seen, both the independent variables (e g , practice) 
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and the dependent variables (changes in behavior) are open .to public 
inspection. This is the essential feature of operational definition . 
call them definitions is to say that they are concerned with concepts and 
not with laws. And, to call them operational is to eliminate everything but 
the reduction bases and mathematical manipulations (Gustav Lerg- 
mann, personal communication). Tbe factual definitions so far considered 
have typically said little or nothing about tbe mathematical relationship 
of learning to its independent variables, being content merely to suggest 
that learning “occurs” or “increases” with practice. More exact defini- 
tion, however, is desirable; and tbe form such a factual definition 
might take may be illustrated by Hull’s (1943) basic definition of learn- 


ing or habit ( S H R ) : 

s H r = M - Me' 1 " 

in which M is tbe theoretical limit of habit growth, e and i are mathe- 
matical constants, and N is the number of reinforcements. This definition, 
thus, is like the other factual definitions in relating learning to practice 
and differs from them only in making a precise assumption about the 
form of the mathematical relationship involved. 


Redefinition 

There are three points about the factual definitions of learning which 
deserve further discussion, especially since they underlie certain basic 
conceptions. These three points involve (1) the learning-performance 
distinction, (2) the empirical fact of reinforcement, and (3) the nature 
of practice. We shall treat each of them in turn. 

1. The Learning-Performance Distinction. Although, as the typical 
factual definition implies, learning often does consist in changes in be- 
havior which occur concomitantly with practice, there are m5y eases in 
which it does not. Suppose, for instance, that one examines a A map for a 
while and then drives directly to another city on the basis of this study. 
If we assume that the act of driving to the city would have been impos- 
sible without consulting the map, it clearly depends upon such practice 
and qualifies as an example of a learned response. On the other hand, it 
is equally clear that learning in some sense occurred prior to the change 
in behavior, and that the trip might have been postponed for a consider- 
able period of time without altering the fact that learning had occurred. 
More technically this means that learning is not, as is so often said a 
change in behavior. Rather it is a change in behavior 'potentiality. Or to 
put it another way, through learning the organism acquires the capability 
to perform certain acts, although sometimes this capability may remain 
Went and its appearance in behavioral change may not be immediate 
Such reasoning leads rather natnrally to the distinction between l earn 
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ing and performance which has figured in most of the influential system, 
ahc accounts of learning. 1 The distinction is chiefly in terms of temporal 
considerations. Learning refers to long-term changes of the organism pro- 
duced by practice. As we have just seen , the products of learning are 
carried with the organism on a fairly permanent basis and are available 
as a basis for action as the occasion demandsl'Perfonnance, on the other 
hand, refers to just this translation of learning into behavior. The level 
of performance depends upon relatively short-term factors such as moti- 
vation, the existence of appropriate environmental circumstances, and 
fatigue. The implicit assumption made fay most writers and the explicit 
one made by Hull (1943) is that learning sets an upper limit to perform- 
ance. Depending upon temporary conditions, therefore, performance may 
provide a relatively accurate or inaccurate index of learning. This last 
point raises a difficult Question for the psychologist interested in learning. 
As we have seen, learning is an intervening variable, an unobservable. As 
such we recognize it and gauge its strength through its symptoms in 
performance. But, as we have also seen, performance may be a good in- 
dicator of learning or a poor one, depending upon other factors. Thus, to 
arrive at a detailed and quantitative knowledge of learning it is necessary 
to study its progress under conditions where these other factors are opti- 
mal, or where their effect is exactly known. Recent years have seen a con- 
siderable advance toward the realization of both of these possibilities. 

2. The Empirical Law of Effect. Turning now to the second problem 
associated with the factual definitions of learning, a careful rereading of 
our sample definitions will reveal that none of them distinguishes ade- 
quately between learning and the loss of learning, or extinction. 2 which 
also depends on practice. The point is that practice alone does not produce 
learning, but only fatigue or extinction. In order to insure the occurrence 
of learning, it is necessary to employ, in addition, the operation of rem- 
f or cement. To paraphrase Skinner (1938), reinforcement refers to the 
occurrence of one of a certain class of events called reinforcers, in the 
proper relation to the to-be-learned response; the proper relation being 
that which tends to increase the probability that the response so rein- 
forced will recur with representation of the situation. The failure to 
reinforce a response decreases the probability of occurrence of the re- 
sponse. Some events possess reinforcing power; others do not. The ques- 

i Tolman who wa 9 chiefly responsible for promoting this distinction, gsre ft ft* 
clearest expression from l.is theoretical point of view in his paper on "performance 
rotors” 0955). In Haifa theory ,H, (habit) is the learning concept; (rml* 
lorn potential) is the major performance concept. Skinner (IM*?), nt one time at 
least made a similar distinction, with reflex reterre repre-cntiDg a learning eoattratt 
and reflex ttrcaeith a performance construct. 

* Technical terms presented without definition In the text are defined In the gio***i7 
at the end of the book. 
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of Tvl.icli events do have it is an empirical one, to be answered by a 
series of d-finin; experiments designed to identify the conditions which 

are effective in producing learning. _ 

The usage of the term reinforcement in the preceding treatment is an 
empirical one. The statement that such events, placed appropriately with 
respect to some response, lead to an increase in the probability that the 
response will recur in the situation is a statement of the empirical law of 
reinforcement (effect), sometimes called the “weak” law of effect. Its 
acceptance does not commit the user to any stronger version of the law of 
effect such as Hull’s drive-reduction hypothesis. For this reason, the term 
reinforcement in the empirical or weak sense is available for use in a re- 
vised definition of learning which takes the two points so far made into 
account: Learning is a relatively permanent change in behavior poten- 
tiality which occurs as a result of reinforced practice. 

3. The Problem of the Nature of Practice. With learning thus rede- 
fined, it will he possible to consider the third problem referred to at the 
outset of this section, although in this case it will not be possible to solve 
it. This problem concerns the term practice. What does it consist of? 
Evidently it does not require the complete rehearsal of the response ulti- 
mately to be performed. Otherwise it would be impossible to learn any- 
thing from maps, demonstrations, books, or lectures (May, 1946). The 
question of just what does have to occur has been a problem of interest to 
•workers in the field of learning almost from the beginning. Attempts to 
answer this question have often taken the form of a search for the neuro- 
anatomical locus of conditioning. If it were possible to find such a locus, 
the nature of practice would probably be considerably easier to describe. 
Using the salivary conditioning experiment described at the beginning of 
this chapter as an example, and without specifying exact neurological 
levels, we might imagine the critical event in practice to be any or all of 
the following neurological mechanisms : (a) the pathways activated by the 
CS (hell), (b) the pathways activated by the UCS (food), (c) the motor 
pathway of the UCR (salivation), or (d) an association pathway connect- 
ing certain of the previous three. Consideration of each of these possi- 
bilities suggests that the neurological effects of practice are essentially 
central and associative. y 

a. The pathway of the conditioned stimulus. Complete interruption of 
the afferent pathway of the conditioned stimulus prevents the formation 
or performance of the conditioned response to exteroceptive stimuli. Sense 
organ stimulation, however, is not necessary, and may be replaced bv 
stimulation of a central projection of the afferent pathwav. To demon 
stratc this fact, Loucks (1934) developed a method for stimulation of tbe 


nerv 


ous system in intact unanesthetized animals. A paraffin-covered coil 
of wire was embedded under the skin of the animal, and insulated J 
were led from it to the particular point to be stimulated. No tires ^nT- 
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trated the skin, and the animal remained in norma] health for Ion" periods 
of tunes. The buried coil could be activated by induction from a primary 
cod external to the animal. With this and similar methods, successful con- 
ditioning has been obtained by substituting f or the CS, electrical stimu- 
lation of the spinal cord, cerebellum, (Gantt, 1937), and a variety of 
cortical areas (Loncks, 1938b; Doty, Rutledge, and Larsen, 1956). 

b. The pathway of the unconditioned stimulus. The sense organ and 
afferent pathway of the unconditioned stimulus are not essential in the 
establishment of the conditioned response as long as excitation reaches 
some central point and evokes an unconditioned response. Successful con- 
ditioning has been produced by substituting for the unconditioned stim- 
ulus, electrical excitation of lumbar posterior roots or dorsal columns of 
the spinal cord (Loucks and Gantt, 1938) and of the lateral cerebellar 
lobe (Brogden and Gantt, 1937). 


The evidence regarding the importance of stimulation by the UCS for 
the performance of an established response is less straightforward. In an 
early experiment by Lang and Olmsted (1923) leg withdrawal in dogs 
was conditioned to a buzzer using shock to the leg as a UCS. Hemisection 
of the spinal cord on the side opposite the trained leg, interrupting the 
ascending pathway of pain fibers, abolished the foot withdrawal response 
to the buzzer although the respiratory response remained. Similarly, 
Martino (1939) reported that the conditioned blink response in dogs is 
abolished by anesthetization of the skin around the eye — the receptive 
field of the unconditioned response. He concluded that a constant stream 
of neural impulses from the unconditioned receptor must be present to 
summate with the excitation from the conditioned stimulus in evoking 
the conditioned response. More recently, Whatmore and Eloitmnn (1916) 
have performed a series of experiments to test the effect of hemidecortien- 
tion on the performance of conditioned motor responses based on shock. 
They discovered that the remoral of the cortical projections of the UCS, 
together with the motor cortex for the responding muscles, completely 
abolished an avoidance response, and concluded that it is the projection 


of the UCS to the central nervous system whieh is important, because 
other responses involving the same muscles survived the cortical ablation. 
Other investigators (Baeva and Rappoport, 1934 ; Yushchenko, Rolle, and 
Pupko, 193-4 ; Settlage and Harlow, 1936 ) failed to find any disruption of 
the conditioned response resulting from interruption of the pathway of 
the unconditioned stimulus. The divergent results remain unexplained 
c. The unconditioned response. Conclitioninp docs not occur if the un- 
conditioned response is directly elicited by stimulation of the motor path- 
way, but only if the excitation is initiated by way of the eentral nernun 
system. If a* conditioned stimulus is repeatedly followed by elect rim! 
Simulation of a motor nerve evoking muscular movement, conditioning 
is unsuccessful (Traey, 1927; Hilgard and M. K. Allen, 1935). Similarly, 
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a TJCR elicited by direct stimulation of the motor cortex in dogs fails to 
appear to the CS, even after hundreds of pairings (Louclis, 1935). In 
much the same vein, if the TJCR is produced by drugs which act directly 
on the response mechanism, conditioning is impossible. The injection o 
pilocarpine causes salivation, by a peripheral effect upon the glands them- 
selves; but no amount of pairing with a neutral stimulus will lead to 
conditioned salivation (Kleitman, 1937; Finch, 1938b). On the other 
hand, salivation produced through the activation of neural centers, by 
morphine injection, is readily conditionable. Such evidence indicates 
quite clearly that the peripheral pathway of the UCR is not essential to 
conditioning. 

Similar conclusions may be drawn from experiments in which the UCR 
is prevented during conditioning. Conditioned salivary responses may be 
established using morphine or acid as the UCS, even if the salivary glands 
are paralyzed by an injection of atropine (Crisler, 1930; Finch, 1938c). 
In addition, it now appears that conditioning occurs under complete 
motor paralysis produced by curare (R. L. Solomon, personal communica- 
tion; see also Black, 1958). This is the strongest evidence favoring 
the assumption that the UCR need not occur for conditioning to be 
established. 

d. An association pathway. The effect of the evidence cited in the pre- 
vious three sections is to push the effects of practice to some central 
region of the nervous system. We have seen, in brief, that ( a ) peripheral 
stimulation by a CS is unnecessary, (6) the response need not occur 
overtly, and (c) the peripheral pathway of the UCS is not a critical ele- 
ment for conditioning, although its central representation may be. Al- 


though such evidence suggests that conditioning involves a central asso- 
ciation process, there is no implication of an anatomically isolable 
“association center.” In fact, available evidence suggests that condition- 
ing is accompanied by widespread modifications throughout the central 
nervous system (Galambos, Sheatz, and Vernier, 1956). 

As regards the definition of learning, the results of the search for the 
neural locus of conditioning simply point up the fact that our definitions 
must contain a measure of ambiguity. The term practice will mean dif- 
ferent things in different experiments; and clarity will come only with a 
more exact knowledge of the effects of practice. 


Theoretical Definitions 

The factual definitions of learning, so long as they remain factual are 
pure abstractions winch do nothing to relate learning to events in’ any 
other universe. In and of themselves, they make no effort to deserfte 
underlying mechanisms or to identify the “true nature" of lea^* 
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Definitions which do attempt to accomplish such objectives are of a differ- 
ent type, which we have chosen to call theoretical definitions. 

Many of the theoretical definitions of learning are physiological. It is 
well known that Hull thought of learning in terns of “ receptor-effector ” 
connections. And Hnnter (1954), having defined learning operationally 
in terms of progressive changes in behavior with repetition, goes on to 
add that “such changes are presumably due to changes in the nervous 
system and are possible because of the plasticity and retentivity of this 
system.” Other examples of physiological theoretical definitions are the 
followings 


... if one afferent (Le. sensory) path already has a connection with a motor path, 
then another afferent path if stimulated simultaneously, or nearly simultaneously, 
with the first will tend to acquire the same motor path of discharge. (Holt, 3931) 

Learning is the process of the formation of relatively permanent neural circuits 
through the simultaneous activity of the elements of the circuits-to-be; such 
activity 13 of the nature of change in cell structures through growth in such a 
manner as to facilitate the arousal of the entire circuit when a component clement - 
is aroused or activated. (Bugelski, 1956, p. 120) 


Another, different, idea which occurs quite frequently in theoretical 
definitions of learning treats the phenomenon as one closely related to 
perception and defines it in terms of a reorganization of the perceptual, 
psychological, or behavioral world of the learner. The theories of learning 
deriving from such definitions are often called cognitive theories. 


. learning consists in the reorganization of a field determined by an obstructed 
need. (Adams, 1931) 

y Learning is a reorganization of the cognitive field. (Krcch and Crutchfield, MS, 
p. 112a.) 


A third conception apparent in several definitions is such as to moke 
learning a modification of behavior already in the organism s repertoire 
of activity. 


'All learning involves, and in reality consists in a modification of instinctive or 
reflex responses. (Franz, 1939) 


Learning tnms out upon anal, -sis to 1* either a ease of dilTcmitial rtrtngtkrning 
of one from a number of responses evoked bp a of wed, nr Ibe forms- 

tion of receptor-effector connections * noro; the first occur, t,T.csr,„.m,.le 
selective learning and the second in conditioned relics learning. (Hull, 1M3) 


Then there is the contiguity proposition adhered to by several theorists, 
that learning is n mntter of developing associations between events occur. 
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ri _ e closely togetler in time. Gntlirie (1938) is. perhaps, most explicit on 
the point l ' Stimulus patterns rriiich are active at the time of a response 

tend, on being repeated, to elicit that response. 

Finally, the idea of reinforcement in the strong sense of the word is 
often a part of the definition of learning. Statements of this sort include 
Thorndike’s classical expression of the law of effect. 


The Law of Effect is that : Of several responses made to the same situation, those 
which are accompanied or closely followed by satisfaction to the animal will, 
other things being equal, be more firmly connected with the situation, so that, 
when it recurs, they will be more likely to recur; those which are accompanied or 
closely followed by discomfort to the animal will, other things being equal, have 
their connections with that situation weakened, so that, when it recurs, they will 
be less likely to occur. The greater the satisfaction or discomfort, the greater the 
strengthening or weakening of the bond. (Thorndike, 1911) 


Obviously the diversity among these theoretical definitions is very 
great. Most of the important systematic issues in the field of learning are 
implicit in them. The conflicts foreshadowed by differences among these 
definitions include those between the proponents of physiological and non- 
physiological conceptions of learning, between cognitive (reorganization 
of the field) and S-R (modification of behavior) views, and between law- 
of-effect and non-law-of-effect positions. Such issues, along with one or 
two others, have provided the impetus for much experimental work in the 
psychology of learning. They are topics which we will meet again. 


The Field of Learning 

Changes of behavior of the sort we call learning range from the simplest 
modifications of the simplest organisms, to the most impressive contribu- 
tions of human intelligence. Learning is basic to the development of ath- 
letic prowess, of tastes in food and dress, and of the appreciation of art 
and music. It contributes to ethnic prejudice, to drug addiction, to fear, 
and to pathological maladjustment. It produces the miser and the philan- 
thropist, the bigot and the patriot, the coward and the hero. In short, 
it influences our lives at every turn, accounting in part for the best and 
worst of human beings and for the best and worst in each of us. Can such 
a range of influence be encompassed by a single concept such as learning? 
As of the present time psychologists are divided on this question Some 
say that all learning is basically the same thing; some say that there are 
two fundamental processes; still others say that there are many 

At the level of operations, it is clear that there are many different 
learning procedures, which have been used to identify research clusters 
such as instrumental conditioning, classical conditioning, maze learning, 
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ring closely together in time. Guthrie (1938) is, perhaps, most explicit on 
the point: “Stimulus patterns which are active at the time of a response 
tend, on being repeated, to elicit that response.” 

Finally, the idea of reinforcement in the strong sense of the word is 
often a part of the definition of learning. Statements of this sort include 
Thorndike’s classical expression of the law of effect. 

The Law of Effect is that : Of several responses made to the same situation, those 
which are accompanied or closely followed by satisfaction to the animal will, 
other things being equal, be more firmly connected with the situation, so that, 
when it recurs, they will be more likely to recur; those which are accompanied or 
closely followed by discomfort to the animal will, other things being equal, have 
their connections with that situation weakened, so that, when it recurs, they will 
be less likely to occur. The greater the satisfaction or discomfort, the greater the 
strengthening or weakening of the bond. (Thorndike, 1911) 

Obviously the diversity among these theoretical definitions is very 
great. Most of the important systematic issues in the field of learning are 
implicit in them. The conflicts foreshadowed by differences amoii" these 
definitions include those between the proponents of physiological and non 
physiological conceptions of learning, between cognitive (reorganization 
of the field) and S-R (modification of behavior) views, and between law 
of-effect and non-law-of-effeet positions. Such issues, along with one 
two others, have provided the impetus for much experimental work in tlT 
psychology of learning. They are topics which we will meet again ** 6 


The Field of Learning 

Changes of behaeior of the sort W e call learning range from the sunniest 
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tions of human intelligence. Learning is basic to the development 
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it influences our lives at every turn, accounting in part for fh u Sh0rt ’ 
worst of human beings and for the best and worst in each of p aud 
a range of influence be encompassed by a single concent supP ^ « 
As of the present time psychologists are divided on this nil t . leanun S» 
say that all learning is basically the same thing; some say thatch ’ **** 
two fundamental processes; still others say that there J there are 
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motor learning, verbal learning, and problem solmnv r„# 
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contiguity, is mo LTL“ 

principles “re required to account for learning under different pro 
cedures The most common position in this latter multiprocess category 
holds that instrumental conditioning is the simplest representative of one 
form of learning, and that classical conditioning typifies the other The 
single-process theorists on the other hand, hold that one of these forms of 
conditioning, because of its simplicity, is, in some sense, basic These 
views of the importance of conditioning set our immediate task for us 
We shall want to look first at the various conceptions of the place of con 
ditiomug in psychology 


SUMMARY 


In order to provide a general orientation to materials to be covered 
later it has seemed wise to begin our discussion of conditioning and learn 
mg with a consideration of the meaning of these two basic terms Tradi 
tionally, learning has been regarded as a more general conception than 
conditioning Conditioning, which includes two subvarieties called classi 
cal and instrumental conditioning is a form of learning which occurs in 
carefully controlled laboratory situations Some theorists believe that all 
learning obeys the laws discoverable in these simple conditioning cxperi 
ments Others feel that additional principles will be required to explain 


the more complex forms of learning 

The more general term learning has been defined in two generally d if 
ferenfc ways Most of the content of this chapter has involved the implica 
tions of these two kinds of definition Definitions of the first sort describe 
learning as any relatively permanent change in behavior which occurs as 
a result of practice There are four points at which such definitions re 
quire comment or clarification 

First, it must be recognized that changes in behavior, which are basic to 
the definition are performance phenomena whereas learning is a hypo 
thetical process which underlies these changes Depending upon other 
circumstances performance may reflect the level of learning relatively 


accurately or inaccurately . , 

Second, unless some reference m made to the process of reinforcement 
definitions of this sort fail to distinguish learning from extinction Rem 
forcement, m one sense, refers to any of a set of conditions which oppro- 
pnatelv employed favor learning In the same sense, reinforcement is 
essential to learning Ua reinforced practice produces only fatigue 
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tinction. For purposes of definition, it is unfortunate that the conditions 
which, qualify as reinforcers are quite varied and appear to fit no recog- 
nizable category beyond their ability to produce learning. This leads to 
an inevitable element of ambiguity in the definition of learning. 

Third, another inescapable source of ambiguity lies in the term practice. 
Efforts to make this term mean anything exact lead to questions about the 
precise events which go to make up a practice occasion. Attempted 
answers to these questions lead to matters of physiology. In the simplest 
form of learning, classical conditioning, it has been discovered that learn- 
ing can take place when stimulation of the central nervous system is sub- 
stituted for the usual external stimuli, and also that learning can occur 
in the absence of an overt motor response. The effect of such evidence is 
to suggest that the critical event in practice is some unknown process in 
the central nervous system, so that, again, it is impossible to define learn- 
ing as precisely as might be desired. 

Fourth, definitions of learning in terms of behavioral changes occurring 
with practice are very close to the level of direct empirical observation. 
Therefore, we have referred to them as operational, or factual, definitions 
to distinguish them from the theoretical definitions also considered in this 
chapter. 

Theoretical definitions of learning either offer some hypothesis regard- 
ing the “true” nature of learning or propose some general condition as 
theoretically essential for learning. These definitions tend to be less closely 
tied to empirical observation than the factual definitions. The theoretical 
definitions are of special interest, however, because differences amorm 
them implicitly represent many of the issues on which theorists of learn- 
ing are divided. 


Fir S t of all we find that some of these theoretical definitions describe 
learning in the language of neurophysiology, suggesting that learning 
must be understood in such terms. Other definitions are completely with 
out neurophysiological reference. Proponents of these theories often lja 
that such speculation is neither necessary nor desirable. a 

Second, some theoretical definitions of learning describe the 
perceptual terms as a reorganization of the world of the learT° C T m 
result of learning according to these definitions, the learner seeT * & 
events (stimuli) relationships which had not previously exists * 

For this reason, these definitions present what has come to he 
stimulus-stimulus (S-S) or cognitive conception of learning A 7 i ' a 

are opposed to S-R definitions, and to related S-R theories they 
learning consists in the establishment of new associations bet^L* 7 lch 
and responses. een stimuli 


Third, theoretical definitions differ with respect to their 
of the nature of reinforcement. Some hold that learning ei ' pretatlon 
when the response to be learned is followed by the satisf ° t ° CCXlrs oal y 
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motile. Others maintain, by contrast, that all that is necessary for learn- 
ing to occur is for some response to be elicited in a particular situation, 
or for the situation to be perceived in a new way Nothing more 
The differences of opinion reflected in these divergent definitions con- 
stitute foci of dispute about which experiment and argument haie cen- 
tered for several decades Many of the experiments to be reviewed in 
subsequent chapters bear directly or indirectly upon these issues These 
research studies in the field of learning have approached the problems of 
learning in many different ways But there is at least one point of agree- 
ment among all students of the learning process, namely, that learning is 
a T,ery pervasive phenomenon, occurring in all of the important aspects of 
human life. 






Conditioning in 
Historical Perspective 

Oxe op the paradoxical facts in tlxe history of psychology is that the 
concepts of conditioning -were accepted with enthusiasm, particularly by 
American psychologists, in spite of the fact that many of these concepts 
rested on infirm empirical foundations (Loucks, 1933). Similarly the 
methods of conditioning seemed to many persons to promise a completely 
objective technique for the study of psychological problems, although the 
validity of the conditioning procedure was never compared with that of 
the established methods. In a sense this evaluation seems to persist today. 
It will, therefore, be instructive to examine the reasons for this uncritical 
acceptance of conditioning for the light such examination will throw on 
the considerations which still govern this judgment. 


PREPARATION WITHIN PSYCHOLOGY FOR THF 
ACCEPTANCE OF CONDITIONING 

When new concepts are incorporated into a scientific system prior to a 
sufficient demonstration of their adequacy, it is apparent that there ex 
isted vuthrn tffii system some kind of preparation or readiness for their 
assumption When such preparation is lacking, important discoveries 
and new methods are often refused admittance There mn<?t i 1Q u 

"7 rr aEnity 

method. Otherwise conditioning would not hare been proposed as a tectf 
mque for measuring sensory thresholds until it wi u 1 a tecll ~ 

the highly refined psychophysical 

basis for learning theory before some effort had been mad! ,o “ “ 
what it accomplished with well-established finding froT^I c™ 1 ” 8 
experiments using problem boxes, mazes skilled t T b ^ PeS ° f 

tionwithintheimmediat^,^h^r^^~ 



‘ conm tioning in historical perspective 15 

prior to tile Pavlovian demonstration The orpins of nsvclioInoIT mW f 
science m general, are m philosophy Although the histor/af psfckZX 
ay bo traced to pre Aristotelian times, the influences most important to 
*5' i” S ° f “« tem PO«r y psychology are clearest in the work 
of the British Empiricists Locke, Hobbes, Berkeley, Hume, Hartley and 
the Mills Their thought set psychology its initial problems, dictated its 
method and, most importantly, determined its rationale Emphasis upon 
the experiential basis of knowledge naturally led to psychology's initial 
interest in the senses and m perception The rationalistic approach fore- 
shadowed the introspective method But, more important for the later 
acceptance of conditioning than either of these were the principle of 
association and the conception of an analytic approach to the problems 
of science 


Association 

Beginning with the assumption that the mind, or consciousness, was 
composed of ideas, the British Empiricists had been led to formulate laws 
of association to account for the relations between ideas The set of laws 
as conventionally stated included three primary ones (contiguity sirai 
Jarifcy, and contrast ) and numerous secondary or quantitative Ians which 
regulated the effectiveness of the primary ones These laws, especially the 
secondary ones, were the antecedents of laws of learning formulated by 
Thorndike and others The laws of association underwent many changes, 
and no one list was ever officially canonized There was a noticeable 
tendency, however, to subjugate the laws of similarity and contrast to the 
law of contiguity, which states that associations are most easily formed 
between experiences that occur close together in space or in time It is 
the one law which was found m all the lists, and it has survived as one of 
the basic principles of conditioning 
In an effort to find a physiological basis for association, the principle 
of contiguous excitation had been proposed, prior to the appearance of 
conditioning on the psychological scene As expressed by William James, 
the principle stated “When two elementary brain processes ha\c been 
active together or m immediate succession, one of them, on rcoccurnng 
tends to propagate its excitement into the other " (W James, 1S50, p 
566) This idea too, has persisted in the thinking of psychologists who 
seek to define learning in physiological terms With reduced 

to contiguous events and stated as a relation betiiccn brain stales, the 
transition to a physiological doctrine of conditioning is straightfonurj 
and does no violence to established modes of thought 
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Analysis 

To understand the easy acceptance gained by conditioning, the idea of 
analysis as a way of approaching scientific materials is at least as im- 
portant as the concept of association. The problem of understanding the 
mind, as the British Empiricists conceived it, was the dual one of analyz- 
ing the content of consciousness into its elements, after the manner of 
chemistry, and finding the principle by which these elements could be 
synthesized to reconstruct mental life. The elements which emerged from 
their analysis of mental processes were ideas described in sensory terms, 
and the principle of synthesis was association. 

Psychology in its earliest versions followed this pattern of thought very 
closely. Consciousness was accepted as the subject matter of the new 
science, and the introspective method was used to discover the sensory 
elements of the mind. These elements were just noticeable differences 
( j.n.d.’s ) in sensations, measured by the various psychophysical methods. 
It was suggested that the laws of association were the principles which 
accounted for the combination of these elementary processes into complex 
conscious experience. 

The notable degree to which the concepts of conditioning could be made 
to conform to this pattern is no doubt a large part of the reason that 
psychology in the second decade of this century found them so appealing. 
Although, in their writing, the behaviorists stressed the contrasts between 
conditioning and the older methods, it has since become clear that early 
behaviorism had much in common with the structuralism it sought to 
displace. Pratt (1948) has presented the important parallels in a way 
-which is reproduced in the table below. It is obvious that, in spite of a 
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Method of psychology 

Experimental observation 
and analysis 
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and analysis 

Results of analysis 
( elements ) 
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diSerence in opinion as to the appropriate subject matter tor psyeholo^ 
there teas a haste agreement between classical psychology and hf>! 
ou matters of general strategy. In a very ° J ? ' ,eha ™risr 

because conditioning followed a traditional sena e, ltj wa Sj a Saii 

so readily accepted. ‘^onal pattern of thought that it ws 

asSiatton ortoZ ’’,7 “ ° bieotors to ™decpt c 

analytic approach to psychology. F rom the ti ^ e 
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tle “T day there have 

the sifitetace o7 hi 7 ^7 ‘ S Wr0ne because Jt Ms “> 

tile substance of h mean Me m anytime; like our firsthand experience of 
it Of such objectors, Koffk-a was one of the strongest Concern ng the 
early form of behaviorism he wrote concerning tne 


Learning could not be reduced to mere mechameal terms Even the questions how 
the selection among different ways of response « to become effective, and what 
factor gradually determines the elimination of useless movements, have been 
answered by [the behavionsts] in the simplest, but also m the crudest, and, as 
"ij a ™ s “ “ a ‘, UraI &elmg for Ilvmg creatures, m the most unsympatbetic manner 
(Koffkn, 1931, p 174 By pei mission of Humanities Press Inc ) 


The answer winch the behavionsts had to this criticism was that there is 
nothing in scientific methodology which requires psychology to concern 
itself with “natural feelings for living creatures,” and that beyond this 
psychology is not required to return from its analysis of behavior with 
an account which matches our naive experience of it, any more than 
atomic physics needs to reproduce our awareness of the physical world 
(Spence, 1948) Whatever the merits of the two arguments, this latter 
sort of reasoning won the day, and the method of analysis of which con 
ditiomng theory is a representative continues to be the more widely 
accepted 


Physiological Orientation 

Although most of psychology recognizes the importance of environ 
mental influences and has certain ties with the social sciences the older 
and more basic branches of tlie field have always remained closer to the 
biological sciences than to any other broad division of knowledge This 
fact, too, contributed to the early acceptance of conditioning Biology 
and physiology had made great strides by 1915 in the study of simple 
reflex activity, but they had been unable to use the same methods w the 
investigation of the complex functions of the brain Since psychologists 
were certain that the physiology most relevant to their problems vis 
bram physiology, tins constituted a serious failure The methods of con 
ditiomng offered a possible way of unravelling the mystery of the brain 
Indeed Pavlov’s avowed purpose m studying conditioning nas to invest! 
gate the activity of the cerebral hemispheres Psychologists accepted both 
his purpose and hxs method Conditioning it was now declared, provided 
11 method of investigating the higher functions winch Justus 

physiological as the methods for studying the spinal reticles 
cause of its pertinence to the physiology of greatest concern to ps, cliologj, 
the conditioned reflex method had \er) " l(lc appeal 
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The Adaptive Character of Behavior 

Although psychology was, by its tradition, prepared for the elemen- 
tarism of the conditioned response and for its brain physiology , classical 
psychology had placed no emphasis upon habit and learning as important 
psychological concepts. Titchener, writing in 1909, preferred not to use 
the word learning in his chapter on association, excusing himself on the 
grounds that learning was such a complex affair. In the following chapter, 
on memory and imagination, he devoted two pages to a discussion of slow 
and rapid learners and to cramming. The word habit was mentioned once 
in relation to the habit of introspection, and again in the chapter on at- 
tention. That is all. Although Titchener was in some sense an assoeia- 
tionist, it is evident from his writings that affiliation with association is 
not enough to force habit and learning into the foreground of a psycho- 
logical system. 

It was another development, Darwinian biology, 1 which brought learn- 
ing into focus as a psychological process. An organism’s struggle to adapt 
to its environment introduced problems with which introspective psy- 
chology was poorly equipped to deal. Whereas classical psychology had 
emphasized the rational, cognitive, intellectual functions of the mind, 
adaptive behavior as described by Darwin appeared unreasoned and emo- 
tional. Such characteristics of behavior resist description in purely con- 
scious terms. Thus the fact that behavior is adaptive required the develop- 
ment of concepts which extend beyond the boundaries of an association 
theory limited to conscious processes. 


As a result of this need, the concept of habit came to the fore as a way 
of combining psychology’s interest in the products of past experience 
with the adaptive function of behavior stressed by the Darwinists. The 
concept of habit was specifically conceived in the Darwinian pattern' Just 
as instincts were regarded as modes of behavior that had been found ad- 
vantageous m the history of the species, habits were considered modes of 

J7 ea k the history of the individual 

( T “ 1 ’ 19 -8>- William James gave prominence to habit in Ms Prin- 

PW early ***“■ 

dearth the fnndament'al 

1 Darwin’s influence on psychologr in v 

himself, was probably unfamiliar with the works or^hf^’ ^ lndireet - Darwin, 
history or psychology, having read neither FecW nor figUres in the 

bj the developing psychology of his time Amn tA 1 ‘ Thus he was uninfluenced 
count the following: (1) the concfptijT of Sas COntri ^tions we 
which ravored a natural-science study of mentel riL ? ^ ° £ evolution > a view 
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processes Indeed he interpreted the orga msm’s capacity to learn b.o 
logical terms, explaining it as a result of the plasticity of protoplasmic 
material The usual introductory discussions of sensation and perception 
he postponed for later chapters James B Angell at the University of 
Chicago gate halnt a central place m his systematic psychology which 
came to be hnown as functionalism Although consciousness remained the 
primary subject matter of psychology, it was argued that the investiga 
tion of adaptive behavior could not always be earned out with introspec 
tive studies, and that mentalistic concepts did not always apply Animals 
had come to be used increasingly in studies of habit where introspection 
was obviously impossible Yet the experiments were undeniably producing 
significant information about the learning process — information winch 
seemed out of place m the framework provided by the concepts of mind 
and consciousness 


The conditioned reflex method was admirably suited to appeal to this 
new trend, for the subjects were animals, the procedures simple and 
straightforward, the results impressive The conditioned response was 
called the unit of habit by psychologists to whom habit was the most im 
portant concept in psychology, the conditioning method was proclaimed 
as a substitute for introspection, once the chief method of psychology, 
and the conditioning process, said a distinguished psychologist plnloso 
pher, brings the mind into being (Holt, 1931, p 28) 


Objectivity 

The study of habit contributed to a general trend in psychology away 
from the so called subjective methods to more objective types of obsena 
tion This trend, begun with the establishment of psychological labors 
tones, with their emphasis on measurement and control, was fostered by 
developments outside of psychology Prom biology came the voices of Vie 
mechanists calling for an objective study of life sciences without recourse 
to mentalistic concepts The famous paper by Beer, Bethe, and V Uexhull 
(1899) suggested new terms for the usual psychological ones, substitnt 
mg for example, reception for sensation Jacques Loeb, the leader of the 
movement, who resided in America from 1392 until his death, taught for 
some years at the University of Chicago, where the influence of hiologj on 
psychology was strong Not without influence was a movement among 
certain philosophers to dispense with the mind body dichotomy and to 
give a more objective account of mind Janies entertained ^ doubts , A out 
tlie need for a concept of consciousness in psychology, ami Holt C > 
attempted a physiological account of the phenomena associated Willi the 
7 A Pennsylvania, Singer (1911) lectured on «<W« « <" 
OhLt And Telman, influenced by the Vienna philosopher,, is carl , o, 
M 22 begin a series of articles in which he proposed a bclinuomlic for- 
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, , t T pa.tment of sucli mentalistic concepts as consciousness and 

mula f ° r „ h emphasis upon physical objectivity was a symptom of the 
S": r JeJe a in the mind ™ re- 

dneed to a reflex of the brain. 


THE ENTRANCE OF CONDITIONING INTO 
PSYCHOLOGY 

In spite of the anticipations of conditioning in the -writings of the 
Greek philosophers, in Locke, James, and others, the history of condition- 
ing, as such, is dominated by the figures of Pavlov and Bekhterev. Its 
tradition is directly traceable to the nineteenth century Russian physi- 
ologist Sechenov. 

Ivan Mikhailovich Sechenov (1829-1905) is properly counted as the 
father of modern Russian physiology ; and, to this day, Russian condi- 
tioning articles frequently begin -with a tribute to him and to Pavlov. 
Trained in the days when Russian scientists did their postgraduate work 
in Western Europe, Sechenov studied first in the laboratory of Johannes 
Miiller in Berlin, and was especially influenced by Muller’s assistant, Emil 
DuBois-Reymond who was, at the time, obtaining evidence for the elec- 
trical character of nerve conduction. Subsequently Sechenov worked with 
Ludwig, Helmholtz, Bunsen, and Bernard. In Bernard’s laboratory he 
performed experiments which provided the basis for his hypothesis of 
the reflex action of the brain (cf. Sechenov, 1935). This hypothesis, which 
set the tradition within which the work of Pavlov and Bekhterev finds its 
meaning, looked upon the function of the brain as a physical reflex, con- 
sisting of three components : sensory input, a central process, and efferent 
outflow. All behavior was to be understood in terms of these material 
processes plus an interaction of excitation and inhibition operating at the 
central link of the reflex arc. This last conception is remarkably close to 
Pavlov’s later theory of cortical function. 


Ivan Petrovich Pavlov (1819-1936) encountered the phenomenon of 
conditioning in the course of his studies of the digestive glands, for which 
he was awarded the Nobel Prize. Briefly, as early as 1880, Pavlov observed 
that sham feeding (in which food eaten by a dog fails to reach the 
stomach, being lost through an esophageal fistula) produces gastric secre- 
tion, just as real feeding does. This suggested that the stimulation pro- 
mded by the taste of food and the act of swallowing it influences the 
stomach leading to a psychic secretion.” Unfortunately the methods 
employed by Pavlov m this first demonstration were exceedingly tedious 
and required a degree of surgical skill which Pavlov, but few others pos- 
seted. I rob ably for this reason, Pavlov turned in his later work to’ sali 
vary secretion, which is easier to measure. 

The first Pavlovian conditioning experiments were very crude. They 
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1 tC cviir' 1 me t a das a piece ° f bread wiiichit ms iaterai - 

at the si -lit of I r 1 &r cmdltl0mn z ™ flat salivat.cn occurred 
P T “*“* Ia tti!se ea ^ «P— h Pavlov 
shou, !f d , 1B P';“ 0nl “ OD ° f emd,t ‘™ed discrmunabon, by 
, lv , 8 tU t a " lmaIs c ° uid be tra ™ d »°t <o salivate at the sight of bread 
dyed some particular color, winch it had not been aliened to eat, although 
tile response persisted to the undyed bread (Brazier, 1959a) Presumably 
ill tins case, the discrimination was actually on the basis of brightness 
since dogs are color Wind ' 

Although Pavlov’s experimental findings were limited to the salivary 
responses of dogs, the concepts which he used m dealing with his data 
were extended by others to discussions of all types of learning Condi 
turned and unconditioned stimuli reinforcement, extinction, irradiation, 
discrimination, and so on, were later employed m analyzing a range of 
topics extending from normal individual behavior to psychopathologie 
and social action In spite of the broad uses made of his work Pavlov 
continued to think of it as belonging to physiology, and he made no at 
tempt to formulate a system of psychology In fact he distinguished be 
tween his work and that of the American animal psychologists because he 
was always interested m the physiology of the nervous system whereas 
they appeared to him to be concerned with essentially human problems, 
even when the } worked with data from animals 
As early as 1886 Vladimir M Bekhterev (1857 1927) had studied the 
localized representation in the dog’s cortex of learned movements such as 
giving the paw, but his systematic study of association reflexes began in 
the winter of 1906 1907 In his experimental investigations which used 
first dogs and later human subjects, he studied the withdrawal and res 
piratory responses evoked by electrical shock stimulation Unlike Pavlov 
Bekhterev was very much interested in creating a new psychology As 
early as 1904, before his work on association reflexes began, ho delivered 
a lecture on “Objective psychology and its subject matter ” lie had 
studied for a time an Wundt’s laboratory, and Ins writings reflect his 
familiarity with the problems of psychology It is probably to him more 
than to Pavlov that we owe the bold acceptance of conditioning by psj 
cholo"ists, although the details of conditioning studios which came to be 
known were Pavlov’s Bekhterev's Objective Psychology appeared serially 
from 1907 to 1912 and was translated into both French and German in 
1913 The German translation carried the subtitle Psycho-Pcjlcxoloyie, 
and Bekhterev’s position gradually became known as reflexologj In 1932 
the translators of a later book were able to declare “Beflevolop is the 
dominant note m Iiuss.au psychology today, mid the Me Billilcrey is 
tlju guiding light of humanistic science tit the USSR The ringe of 
Bekhterev's interests was enormous He wrote widely (COO titles) on 
neurology psychiatry, infancy, education, social psychology, judicial 
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psychology, and aesthetics, and he founded a number of institutes for the 
study of problems in various areas of neuropsychiatry and child study. 
Among his contributions to the subject matter of psychology -which found 
at least temporary acceptance, were (1) methods of conditioning leg and 
finger flexion and cardiac and respiratory responses, (2) the concept of 
thinking as subvoeal speech, (3) the idea of using conditioning for test- 
ing sensory thresholds, (4) the interpretation of concept formation in 
terms of generalization, (5) a fairly clear statement of a stimulus-sub- 
stitution theory of conditioning, and (6) a denial of introspection as a 
fundamental psychological method (Bekhterev, 1928). 

The 1913 translations of Bekhterev’s book were very important in 
arousing the interest of American psychologists, and most of the Ameri- 
can experimenters preferred to use his motor conditioning methods rather 
than the salivary method of Pavlov. Nevertheless, both Pavlov’s ter- 
minology and his conceptual system were more widely adopted than 
Bekhterev’s. It may be that Bekhterev’s term association-reflex carried 
with it too much of the flavor of the association of ideas, and thus seemed 
to continue a tradition against which behaviorism was reacting. This 
might be one reason why Pavlov’s term conditioned reflex was more ac- 
ceptable. Moreover, Pavlov’s careful working out of one variety of ex- 
periment with great precision gave his results greater usefulness to 
laboratory workers than Bekhterev’s more general applications. 


Early Interest in Conditioning in America 

(t rm avl ° V ' S 1906 IIuxley le cture, delivered at Charing Cross Hosnital ™ 

. ie scientific investigation of the psychical faculties or processes in’ the 
higher animals appeared in Science and was thus made available to the 
American public. Details of the experiments from hie mi™ abie to the 
summarized for presentation to psychologists in a thn l b , ° iy were 
Yerkes and Morgulis in 1909. The review f tb °/ 0Ug h revi ew by 

immediate repetitions of Pavlov’s work in A ° WeV . er ’ dld not lead to any 
records reveal. merica, so far as published 

Krasnogorski, a student of Pavlov nerfrw™ a 
periments on children, reported in German in 1 1 ^ Condltioni ^g ex- . 
came to the attention of W H Burnhe + n^ 9 ^ 111 1913 ‘ These 
corporated the material into the lectu™ £ hie wl “ in- 

hygiene of the school child. Amon- his stnrl + bis ™ urse 011 the mental 

Mateer, who became interested Tn reW ^ 19134914 was Florence 
devoted the next two years to work in SenTT ^ WOrk ' Ske 
into a doctoral dissertation in 1916 and tuhh l J * Was incor P°rated 
Mateer s experiments constitute the ™ P 1 bed m book f orm in 1918 
country. She used the °* «* 

Cjcb o£ the child just before food was placed in T S “ bandage ovtT tie 

P aced m hls “tenth. The bandage 



CONDITIONING IN HISTORICAL PERSPECTIVE 23 

came to serve as the conditioned stimulus, eventually evoking eliewin" 
and swallowing movements before the food was presented The move" 
ments were recorded liymograplncally f rom tambours fastened under the 
c m an on the throat The use of the bandage as a conditioned stimulus 
was discovered more or less accidentally Mateer writes 


Tfae ffrcat significance of the method came to me all at once about the fourth or 
tilth day of my first experiments with Phil, m 1914 I learned that even acceptance 
ol a teat posture, or entrance into the experimental laboratory, was a conditioning 
factor and that these and other casual environmental factors had to be imcondi° 
tioned through disuse before any arbitrary conditioning factor might be used as 
predetermined in a planned procedure Even with babies who could not sit up, 
the bandage was a conditioning factor as valuable as other stimuli in evoking 
response Neither Bekhterev or Krasnogorsh prepared me for this, and, though 
I had read Pavlov, it took personal experience to show how significant the 
minutiae of an experimental setting must be 


Mateer ’s successful conditioning of children furthered the case which 
Burnham made for the place of the conditioned redes in mental hygiene 
(1917, 1921, 1924) 

The appearance of the translations of Bekhterev’s Objective Psychol 
ogy brought conditioning strongly to the attention of psychologists and 
references to at began to appear m 1916 H C Brown (1916) pointed out 
a possible connection between language and the conditioned reflex and 
P L Wells (1916) suggested that conditioning could supply aeon* enicnt 
account of affective transference 


Watson’s Influence 2 

As we have seen, the preparation within psychology for the acceptance 
of conditioning extends hundreds of years into its history The most ira 
portant elements m this development had been (1) associatiomsra (2) 
the analytic approach to science (3) a physiological orientation (4) tho 
habit concept, and (3) objectivity The comergence of these die historical 
influences gave rise in America to Behaviorism launched by John B 
Watson m 1913 Watson s theory was related to all five trends It was an 
objective association psychology, with an cmjiliasis upon habit biological 
in viewpoint and analytic in approach Initially it did not, hO'icscr place 
great importance upon the concept of conditioning Prior to his pres 
dential address entitled “The place of the conditioned relic* in ps S cl ol 
ogy” (1916a), which was delivered in 1915 before the American Is clm- 
Wal Association the conditioned relic* concept had little 
American psychology After tins date, it occupied an increasingly prom, 

■ Fer a thoughtful audj.u, of r>*» - >“‘°V ” 

Bergmann (19SG) 
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nent place in the textbooks, if not, for a time, in the research laboratory. 
In recounting his relation to Pavlov, Watson writes: 

I had worked the thing out in terms of Habit formation. It was only later, when I 
began to dig into the vague word Habit that I saw the enormous contribution 
Pavlov had made, and how easily the conditioned response could be looked upon as 
the unit of what we had been calling Habit. I certainly, from that point on, gave 
the master his due credit. (Personal communication) 


During the winter of 1914-1915, while Mateer was working independ- 
ently at Clark University, Watson’s seminar at The Johns Hopkins Uni- 
versity was devoted to the translation and discussion of the Erench edition 
of Bekhterev’s book. In the spring of 1915 a number of Bekhterev’s ex- 
periments were repeated, chiefly by Lashley who was then working with 
Watson; and experiments on finger withdrawal, respiration, and° heart 
rate provided the factual substance of Watson’s presidential address. In 
this presentation Watson exploited conditioning primarily as a substitute 
for introspection. He sensed its importance, however, in relation to emo- 
tion, mental disease, language, and habit, and wrote about these matters 
within the next year. 

The andtaaud response was prominent in Watson’s textbook (1919) 
although it had not yet completely displaced other concepts such as 
instinct. But the way had already been paved for the ^nlonoTLdl 

tmnmg concepts into mental hygiene (Heilbronner 19 W • W a tL i 

Wells, 1916; Watson and J. J. B. Wan. laiy i l ,?’ 1916b; 
Watson’s last experiment (with Rosalfe Rayner) W9 W as ? ’ **? 
suggest important applications to child psychology This 1° 

with the boy Albert, whose fear of a loud sound 77 es Penment, 

rat and then to other furry obiects nmh hi S C ;° ltloned to a wllite 
single case in the conditioning literature* Z ^ most famous 

haviorism (1925), conditioning was the central «, atSOn . S book ’ Be ' 
he espoused a radical environmentalism and ais^Td There 
forms of inherited tendencies. The remaining mStlnets and other 
the unconditioned reflex; the unit of 1 ° f natlve behavior was 

’ * ° f bablt Was the conditioned reflex. 

Conditioning, 1916-1929 

During the first ten years after Watsrvn »<. 
cept of the conditioned reflex gained Z JZZ ^ address > tke con * 
there were very few psychologists performtf ^ im P 0 rtance, but 
mg Mateer s experiments, completed in lg ° eXperHnents on condition- 
and Cason (1922a, 1922b) reported 7, 19 6 ’ ^ ere Polished in 1918 

p diary and blink reflexes 3 But on th SUCCeSS in COI ^itionin- the mi’ 
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the salivary response in man Hamel (1919) concluded from Jus studies 
o£ Huger withdrawal that conditioning in man was confused by voluntary 
processes but neither his monograph nor the paucity of new experimental 

the hir / , e "r, 8 Me 0f P°P uIant y » £ <*= conditioned reflex as 
the basic idea employed by a number of psychological theorists The first 
general textbook to use a conditioning concept throughout was that of S 
►smith and Guthrie (1921) They adopted the term conditioned response 
instead of conditioned reflex, broadening the conditioning concept to in 
dude all that had formerly been treated as associative learning Allport 
(1924) first gave prominence to the conditioned reflex in a textbook of 
social psychology at a time when social psychology was seeking desper 
ately for a substitute for instincts. Burnham’s Normal Mind (1924) p re 
seated an extended review of conditioning literature in a textbook on 
mental hygiene In the later twenties, textbook writers m psychology, 
sociology, and education allotted increasing space and prominence to the 
subject 


In 1927, the Anrep translation of Pavlov’s Conditioned Reflexes ap 
peared, making a comprehensive treatment of Pavlov’s work available in 
English for the first time and leading to a strengthened interest in ex 
perimentation on the conditioned reflex This volume made mailable in 
rich detail the facts discovered by Pavlov and his co workers during more 
than a quarter of a century devoted to the study of conditioned salnation 
in dogs Pavlov’s experiments and theories called for repetition and con 
Urination, using other organisms and other responses Many laboratories 
now followed the example of the pioneering few and initiated programs 
of conditioned reflex experimentation In 1926, Liddell began a series of 
conditioning experiments on several different animals Kleitman and 
Cnsler's important study of conditioned salivation to morphine injection 
appeared in 1927, Schlosberg’s study of the conditioned knee jerk was 
published in 1928, and Upton’s study of hearing in the guinea pig 
(1929) was followed by a series of investigations by others on the sensory 
capacities of animals lathe next few years experimental reports an con 
diiioned responses appeared in rapidly increasing numbers and the use- 
fulness of the method for the study of learning, sensory acuity and re- 


lated problems was clearly demonstrated 

Although the laboratory importance oi conditioning was on the in 
crease during the late 1920 's, the textbook treatment of the topic had 
begun to decrease This was because of a waning confidence in the nar 
rowly formulated behaviorism of earlier J ears ami the emergence durmg 
tins period of several strong competitors for tbe theoretical 
of psychology Kohler's Me Vestal, l<r of J pcs, oppeinug m English in 
ill broZZ the concept of insight dramatical!) before animal ex,™ 
menters Yerl.es had alread) been promoting similar concepts and bad 
u ver accepted behaviorism In spite of his great respect for lavlov, he 
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wldeh behaviorism bad been strong, such as child psychology, to become 
more eclectic. Koffka’s The Growth of the Mind (1921) emphasized ma- 
turation, -which, as a hind o£ substitute for certain aspects of grown 
formerly included in instinct, now became a formidable competitor to 
conditioning as a factor in the development of the child s behavior. 


Conditioning, 1929 to the Present 

Perhaps the most significant development in the decade following the 
appearance of Anrep’s translation of Conditioned Reflexes was the emer- 
gence in Am erica of a number of theories of learning. In this develop- 
ment the conditioned reflex experiment assumed a new role. "Whereas it 
had been the object of study in its own right, it now provided the con- 
cepts which Guthrie, Hull, and others applied to more complicated learn- 
ing. The part played by conditioned-reflex work was central in some of 
these theories and secondary in others. 

In 1929, Clark L. Hull, at Yale, began a highly provocative series of 
articles in the Psychological Review making use of the findings of Pavlov 
and of other workers with the conditioned response in a manner not pre- 
viously attempted. His presidential address before the American Psycho- 
logical Association in 1936 (Hull, 1937), two decades after "Watson’s, 
shows the changes in the use of the conditioning experiment. Hull, like 
"Watson, stressed the conditioning process but, in the light of the new 
detailed knowledge of conditioned responses, he presented a system of 
definitions, postulates, and theorems, by which to bridge the gap from 
conditioning to more complex forms of learning. 

Hull’s original article, entitled “A functional interpretation of the 
conditioned reflex,” bore the stamp of Darwinism and set a general theme 
which characterized Hull’s work to the end of his life. One of the first 
questions Hull tended to ask about any behavioral phenomenon concerned 
its biological utility or survival value. Later articles dealt with such com- 
plex phenomena as trial-and-error learning (1930a), knowledge and pur- 
pose (1930b), problem-solving (1935c), and serial rote learning (1935a). 
The^e systematic formulations led to numerous experiments by Hull’s 
own students and by others. They also led to efforts to work out theoretical 

si sterns to handle still other learning phenomena using conditioned-re- 
sponse concepts. cu 

In all of bis efforts at system building, Hull attempted formal rl-or 

reSTbis’ tetter* 8 “Y m ° del “* NKrtoni “ Panics. He 

bis method, somewhat inaccurately, as “hypothetieo-dedue- 
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2L-- f ° r hmiseU the *>« criteria as necessary for 

adequate theorizing m psychology 3 * 


JJ„ b yT'T a “ d P ° Sfl,kl ' S 0f a smatlSc «!*• should be stated m a 

ttlv sbould he r°T ““T’ J Si ‘°“ W be C0ns,st “ t mth one “»«"»> aid 
they should be of such a nature as to permit rigorous deductions 

, Tie labor of deducing the potential Implications of the postulates of the 
system should be performed with meticulous care and exhibited, preferably step 
by step and in full detail It is these deductions which constitute the substance of 
a system 


III The significant theorems of a truly scientific system must take the form of 
specific statements of the outcome of concrete experiments or observations. The 
experiments in question may be those which have already been performed, but of 
particular significance are those which have not previously been earned out or 
even planned It is among these latter, especially, that the crucial tests of a 
theoretical system will be found 

IV The theorems so deduced which concern phenomena not already known 
must be submitted to carefully controlled experiments The outcome of these 
critical experiments, as well as of all previous ones must agree with the corre 
spondmg theoiems making up the system (Hull, 1935a, p 49a 496) 


The high point in, Hull’s senes of efforts at system building appeared 
in 2943 with the publication of his Principles of Behavior In this booh he 
returned to a consideration of simple learning and presented a deductive 
system to account for the major phenomena of classical and instrumental 
conditioning One of the outstanding characteristics of this theory was a 
heavy reliance upon Thorndike s law of effect, interpreted as drive re 
duction This feature of Hull’s theory had played a very small role m the 
earlier miniature systems but now assumed a position of central impor 
tance The other major distinguishing feature of Hull’s system was the 
hypothesis that learning is a gradual process rather than a sudden or 


insightful one 

Principles of Behavior was hailed by some (Koch 1944 ) as providing 
evidence that psychology had finally reached a stage of maturity where 
rigorous system building was at least worth trying By others (keeper 
1914 ), it was criticized as the culmination of a series of unfortunate 
trends in psychology The first of those evaluations won out, and Hull’s 
theory directly or indirectly dominated research in the field of learning 
for the nest decade In a lesser degree it continues to do so After Err i 
note of Behavior, Hull published two more books (Mol, Wo.) «i winch 
he described major rev.sions in Ins thinking Some of these tensions Ime 
been accepted by Hull’s followers and worked into sRll further ren-ions 
the system (Spence, 1936) Others hare not met with genera! accept 
1 . » Hull K Iheorr I ate been subjecteJ to analjtt bf 

( Z): «!£rV», «”<”*» <«»>. *•“ **' 

JJunham (2 S>j7) 
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an!! and, as a strictly Milan system, tire 1943 version remains the most 

lIc'deLtue o£ the thirties saw, in addition to the development of 
HuUian theory, the appearance of three important rival positions. In 
1935 Guthrie published the first edition of his Psyclwlogy o) 
which proposed what has come to he called a non-reinforcement \ ie\ 
learning In this hook Guthrie developed the thesis that all that is neces- 
sary for learning to occur is for a response to occur m a given situation 
and for that situation, then, to change. The response, in one such oc- 
currence, becomes conditioned to the stimuli in the situation and can be 
eliminated only by the conditioning of some new response to the same 
cues. This position differs from that of Hull in two important respects: 
m ^ learning is regarded by Guthrie as a sudden rather than a gradual 
process and (2) the mechanism of reinforcement is stimulus-change 
rather than drive reduction. On the other hand, there are certain points 
of agreement between Hull’s theory and Guthrie’s on matters of strategy : 

(1) both theories assume that learning consists in the development of 
stimulus-response associations or connections and (2) both imply an ap- 
proach in which the consequences of certain basic premises are arrived at 
by a process of deduction. Experimental tests of these consequences then 
may he taken as support for or argument against the theory. 

Another important psychologist who presented the first detailed state- 
ment of his systematic position in this period was B. E. Skinner. Skinner 
agreed with the S-R approach but rejected many of the other features of 
prevailing theorizing. In his book, The Behavior of Organisms, published 
in 1938, Skinner described the results of a series of experimental studies 
of the bar-pressing behavior of rats, begun in 1930, 4 and offered a general 
approach to psychological investigation which differed sharply from that 
of Hull and Guthrie. Eor one thing, Skinner made a distinction between 
classical (respondent) and instrumental (operant) conditioning and held 
that the laws of one were not the same as the laws of the other. For an- 
other thing, Skinner objected to the conception that the essential pro- 
cedure in the development of scientific understanding is deduction He 

was particularly opposed to the hypothetieo-deductive method advocated 
by Hull: 

A quite different emphasis is to bo found in the preceding chapters. Deduction 

° £ hs '* U ‘“? ”» subordinate to the quantitative determina- 

Ir Ue'diZto” ! ,™ r a ° a thr ° nsl ‘ i ” d “ ti »“ «* establishment of 
lan=. The d, faculty seems to he in the model Hull has chosen. A science of be 

humor cannot be patterned after geometry or Newtonian physics Teca™' its 

physics or chemistrv as a model the place of d 'l f \ Ch ° Sen 

been much less important. (Skinner, 1933, p 437) “ W 

• For eiamplc, Sbumer, 1030, 1934, 1935,, 1935b, 1936,, 1936b, 1936c, 19364, 1936c 



CONDITIONING IN HISTORICAL PERSPECTIVE 29 

In place of , hypotbet.c'Hleductive explanation, Sbnner proposed a 
P rely behavioral system and argued against the use of neural hypotheses 
to account for behavior He put his position strongly 


The gain to the science of behavior from neurological hypotheses m the past is, 

V f f ° ut,,H S M b f ^ misdirected experimentation and 

tles3 theorizing that have arisen from the same source {Skinner, 1938, p 426) 


Beyond this, Skinner studied behavior in a free responding situation 
whereas almost all psychologists m the field of learning employed trial by- 
trial procedures , he was most interested m a problem (intermittent re- 
inforcement) which had dot yet caught the imagination of American 
psychologists, and he presented his data unorthodox!/ in the form of 
kymograph records produced by the rats pressing the lever in the Skinner 
apparatus For all of these reasons, Skinner’s view gamed few adherents 
until relatively recently Now, nearly twenty five years after the pubh 
cation of The Behavior of Organisms > the full impact of Skinner theory 
is finally being felt 

A final influential figure contributing to the psychology of learning in 
the 1930's was E C Tolman, whose major statement appeared in his book, 
Purposive Behavior tit Animals and Men (1932) Tolman ’ s influence upon 
psychology was partly through the development of a theory and party 
through his treatment of methodological matters Tolman ’s theory differed 
from that of the other theorists discussed here m that be was a eogmtne 
theorist Learning, for Tolman, consisted in the acquisition of bits of 
knowledge, which he called sign gestalts, about the environment 


Sign gestalts are to be conceived as having three parts a sign a sigmpeate, and a 
behavior route leading from sign to sigmficate A sign gestalt is equivalent to au 
"expectation” by the animal that “this,” (that is, the sign) “if beba\cd to in such 
and such a way” (that is, the behavior route), will lead to 4 that” (that is, the sig 
mficate) A sign gestalt sets the animal to “expect” that when he actually behaic«, 
“this field feature wJJ lead, ‘ by such and such a behavior route” to ‘ that” field 
feature And learning, according to the theory, will consist in the making or the 
remaking of such expectation sets (E C Tolman, in F A Voss, Ed, 

Comparative Psychology, 1934a. Prentice Hall, Inc, Englewood Cliffs, N J) 


This passage marks Tolman as one who beheics in learning as the 
acquisition of expectancies, or knowledge about "what leads to wliat, in 
the environment The position is one which has been called Hie «««»«• 
stimulus (S S) view of learning Although there is nothing in the S S 
position which demands it, Telman’s theory is also a non reinforcement 

th To7n,an also influenced the psychology of learning sen , “ 
basic, methodological ways Ho was responsible for the fin dm -I 
ment of the learning performance distinction lie was one of the first 
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told that conditioning was a limited form of learning. And, in a series 
of articles begnn in 1922, he brought the concept of the intervening vari- 
able to American psychology. In these articles Tolman (1922a, 1922b, 
1922c, 1923, 1926, 1927) made clear a fact which his anthropomorphic 
terminology tends to obscure, that his position was a behavioristic one, a 
position which he, himself, called an operational behaviorism (Tolman, 
1936b). 


Current Trends 

Although emergence of major systematic positions was the most im- 
portant single development in the field of learning during the decade 
following the appearance of Anrep’s translation of Pavlov’s Conditioned 
Reflexes, there were several other trends worth mentioning. 'With the ap- 
pearance of relatively definitive systematic positions there occurred dif- 
ferences of theoretical opinion which led to crucial experiments, polemic, 
and more crucial experiments, concerned most often with the principle 
of reinforcement, the question whether learning is gradual or insightful 
and the controversy implicit in the S-S versus S-R interpretations of 
earning. Neither polemic nor experiment, however, dissuaded the major 
theorists, and it gradually became clear that the theories were so complex 
so ambiguously formulated, or both, that none of them could possibly be 
disposed of by a single empirical or dialectic coup de grace. This led to 
a lessening of interest in such attempts. 

Instead, more recent efforts have been mar] P tn ^ 

applicability of certain of the theories. Hull’s tb P mJ Pl ^ range ° f 
for accounts of verbal learning CRVin i y Was ma( ^ e the basis 

(Miller and Dollard 1941) motor ^ -° 194 °^ soeial Savior 

*»>■ a n<i -Ti A sr T r ! .r e ' 

Skinner’s theory found important appliJabilit be - UU ' 10r (Guthrie ’ 1938 ) • 
animal training (Breland and Breland 19511 * P ractlc al problems of 

tion in the ‘°The mii ° r °* dil ’ ee - 

current scene is a systematic stakdsSwir aetW V m 
t eornts are attempting to consolidate such 19 “ 9> ’ m " Ucl1 tl,c 

search mat. gated by the theories of the mv T Were ma<5 c from re- 
possible, new facts and interpretation S," 4 t0 borate, where 
of sources. Although it is always riS4 COme a numbe, 

most recent trends in a field 'the f„u mpt to P°>nt out the yen 

most important : the foffowmg few seem to he among the 
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J , “f that the great theories developed in the 1930’t, 

d Leti^ ? * ° a SrStema “ C “ d fMtol «™»* p“ 0 f 

deduction w playing a smaller role than it once did m the psychology of 

earning In its place there has appeared an increased emphasis on “fact 
n mg parametric studies Where the learning psychologist once had a 
theory lie now has only a “point of view ” 

2 There has been a great strengthening of interest in the biological 
aspects of psychology New and exciting discoveries relating behavior to 
neural and endocune functioning are leading to important modifications 
m our conceptions of motivation and reward And the research of the 
ethologists has brought a revival of interest in instinct and other forms 
of inborn behavior 


3 The phenomenon of attention no longer seems one which can be 
omitted in the description of even the simplest behavior Work on the 
reticular formation by the neurologists and physiological psychologists 
has provided a physiological basis for arousal and attention Behaviorally 
there is a greater interest than formerly m observing responses (WyckofF, 
1952 1954) 

4 The concept of behavior as always m some measure, mediated has . 
taken on increased significance (Osgood 1953) It now seems clear that 
the S It formula however powerful in the hands of a Hull or Skinner is 
at best incomplete There must be a recognition that a great deal goes on 
between S and R 

5 The great complexity of human behavior is currently a matter of 
stress Accumulating evidence shows clearly that even the simplest condi 
tioned response is controlled by a great many variables Furthermore 
these variables interact in ways which mean that a condition which has 
one effect under a certain set of circumstances may have a different e\en 
opposite effect under another 

6 Conditioning procedures both classical and instrumental are more 
popular today than at any time in the past Two mam factors seem to be 
contributing to this (1) Improved apparatus makes excellent stimulus 
control automatic scheduling of trials and detailed recording of re- 
sponses possible and (2) closer contact with Russian research mamly in 
physiology has brought the fruitfulness of conditioning methods again 
to the attention of psychology 


THE PLACE OF CONDITIONING IN PSYCHOLOGY 
Different writers tale made a .anety of interpretations of tta place 
of conditioning m psychology Pavlov, himself, believed that hn i method, 
were purely physiological, and Ins aim in conducting Ins conditioning 
studies vasT entirely that of gaining an understanding of tta functioning 
ofUieccutral nervous system Similarly, his .beor.es were expressed in 
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tlie terms familiar to the neurophysiologist. The extension of Pavlov’s 
work in Am erica more typically consisted of attempts to formulate a 
theory of learning in conditioning terms. Naturally such applications 
sometimes reflected opinions -which were very much at odds with those 
of Pavlov. 


Pavlov’s Theory of Cortical Function 

The fundamental conceptions in Pavlov’s physiological theory "were 
those of excitation and inhibition, conceived as cortical processes. 5 Ac- 
cording to the theory, afferent stimulation produces an excitatory process 
at a definite point on the cortex from which it gradually “irradiates” or 
eads over the entire sensory area or “analyser,” diminishing in in- 


du a w-cj-mLiLc jjuj-LiL un uic uuiLc-A nom which it graclu 
spreads over the entire sensory area or “analyser ’ 
tensity with distance from the primary point. 

The mechanism of the formation of simple conditioned responses was 
explained by Pavlov as follows. The excitation initiated by a neutral 
stimulus at point A irradiates over the cortex, and will be concentrated 

° T tat r 10n ^ ar0USed ^ a * ^conditioned 

stimulus After a number of repetitions of the two stimuli, the excitation 

aronsea by the neutral stimulus is drawn to the locus of the Zcondi 
troued stnnulus m sufficient intensity to elicit the conditioned rcspo^ 
The direction of drainage of excitation is from the weaker to ti 1A + P 
oi : mor e dominant t focus of excitation. The 

(p. 8/) 6 was predicted from the assrnnnHn-n tp + -a. • ° enerallzat1011 

aroused by a stimulus similar to T f ^ &t Pohlt 

irradiate widely and be drawn into the Litable f!T ^ 77^ 
tioned response, either directly or by way of point A T* °* ^ * ncondi - 
guished (p. 82 ) bv presenting it* c+t p mt pomt B is extin- 
proeess at this point is changed from _ as vrit hout reinforcement, the 
irradiates ove/the ZTrtZ ‘° ^ 

provides a means whereby nonreinforcp™ + eortex - This mechanism 
a response to stimuli other than the trainin'! ' !* ay reduce tlle strength of 
Aization of extinction (p. 92 ). The phase of t^T^ 7 ****** tlle gener- 
Pavlov to he Mowed by a receion or “ lrtadlati “ ™ assumed by 
to its point of initiation, Ae entS sequence of ° f 

Iron requiring £r„ m a few second )™™ o£ “radiation and concentra- 
In actual experimental situaW IT ^ ™ ntes *° oceur. 
e.tation was detected by conditioning a re^poSTt ° f and ex- 

altematively extinguishing an already Sinned or 

theory is 4e<cribrf . , y conditioned response, and then 

— <—, l902 , 1927 , 

oa the pages listed in parenthSeT * Processes trill find 
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°ta"™v],!ch wte a°LmedTo a br nt Tn°“ * nei * ib °™« 

ro UgIl geometric “ 

ing example of Pavlov's (1927, p 165-166) method „f ”em g the fpreTd 
and recession of cortical effects involves the inhibitoiy mechanism Sena 
rate and equally strong conditioned responses mere first established^ 
Uiree different tones and a hissing sound Then the response to one tone 
was extinguished and tests were made with the other stimuli at times up 
to fifteen minutes later Figure 2 1 shows the results obtained when the 



FIGURE 2 1 Pavlov's evidence for the irradiation and concentration of m 
lubjtion Following the conditioning of a salivary response to three tones 
and a hissing sound, the response to the tone of lowest pitch was extinguished 
Tests were, then, performed with the other stimuli Note that the extent to 
which the CR is inhibited is greatest for the most similar stimulus and that 
the effect increases and then decreases m lime Data from Pavlov (192?) 


extinguished tone was of 123 cycles per second and the test stimuli were 
tones of 132 or 1161 cycles per second or the hissing sound Two points of 
importance appear in Figure 21 (1) The extinction earned out with 
the low tone produces a diminution of response strength to the other 
tones by an amount which depends upon the similarity of the stimuli 
This is the familiar process of the generalization of extinction (2) The 
amount of inhibition increases at first and then decreases This is Pal lot s 
evidence for the initial irradiation and subsequent concentration of the 
inhibitory process 

Pavlov assumed that the production of excitation or inhibition in nnj 
cortical center automatical!} produced the opposite process m s™”' 1 ' 
ing areas, and that as the cortical process subsided, it might be succeeded 
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by tie opposite process. Pavlov called these effects positive and negative 
induction and referred to their behavioral consequences (p. 93) t 
same terms. Thus negative induction referred to the intensification of in- 
hibition under the infiuence of excitation ; and positive induction ref ene 
to the intensification of excitation under the influence of inhibition. 

The chief environmental variable thought to determine the intensity o± 
the cortical process, and therefore the degree of irradiation, was the in- 
tensity of stimulation. This hypothesis was based upon the fact that, as a 
general thing, Pavlov obtained a marked positive correlation between the 
intensity of°the conditioned stimulus and the magnitude of the condi- 
tioned response (Pavlov, 1930). Eventually, however, it became neces- 
sary to postulate two different physiological variables to account for the 
fact that certain other conditions also affected the magnitude of the 


response. 

One of these additional variables was the excitability of the cortical 
cells. It was necessary to assume different levels of excitability to account 
for the facts that mild hunger and small doses of certain drugs such as 
caffeine tended to increase the magnitude of the conditioned response and 
certain other conditions such as satiation and old age tended to decrease 
it. Other evidence gave rise to the second new concept, that of the “limit” 
or top capability of the cells (Konorski, 1948, p. 27). Specifically, ex- 
ceedingly strong stimuli sometimes led to a destruction of the normal re- 
lationship between CS strength and reflex magnitude and produced 
responses considerably smaller than those elicited by a CS of moderate 
strength. To account for this, Pavlov assumed that the cortical cells re- 
spond to increasing CS intensities with increasing excitation only up to 
their top capability. Beyond this point, the CS produces a protecting 
inhibition which reduces reflex magnitude and prevents damage to the 
cortical cells from overstimulation. 


The top capability of the cells was assumed to vary from animal to 
animal; such factors as old age, neurosis, and fatigue were believed to 
lower it. Lowering the top capability of the cells theoretically could have 
any of a number of effects, depending upon the degree to which the top 
capability was lowered. For instance, a strong CS which had previously 
evoked a very strong response might, with a modest lowering of the top 
capability of the cells, now produce the response typical of a weak stim- 
ulus. This was Pavlov’s phase of equalization. Still further lowering of 
top capability produced a paradoxical phase, or even an ultra-paradoxical 
phase. In the first of these, only weak conditioned stimuli can evoke a 
l espouse. Strong stimuli either evoke no response at all or have a barely 
no iceable effect. The ultra-paradoxical phase, indicative of a still further 
loucrmg of the top capability of the cortical cells, is one in Si 
inhibitory stimuli have a positive effect. The explanation for this is that 
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the completely inhibited cell, receiving the inhibitory impulse „ mad „ to 
respond with excitation through induction 


Evaluation of Pavlov’s Theory 

The outline of Pavlov's theory sketched above shows that it contained a 
sufficient number of concepts to produce an elaborate theory to explain 
the operation of the cortex This was Pavlov’s aim, and he and h,s col 
leagues tried very hard to construct such a theory Moreover, Pavlov was 
iully aware of the complexities inevitably to be encountered in brain func 
tion He regarded the cortex as capable of responding to sensory stimula 
turn from all parts of the body and believed that the irradiation of the 
cortical representations of these effects and their inductive interactions 
with the CS would have to be taken into account in order to explain the 
functioning of the cerebral cortex 

Toward the end of his life, Pavlov also began to develop Ins theory in 
directions which suggested the possibility of dealing with important psy 
chological problems in conditioning terms Sleep was considered the re- 
sult of extreme irradiation of the inhibitory process over the entire cortex 
and into the lower brain regions Hypnosis was a form of partial sleep, or 
less extensive irradiation of inhibition Neurosis was regarded as a func 
tioual pathology of the cortex in which, as a result of an unusually acute 
clashing of the excitatory and inhibitory processes or of the influence of 
strong and extraordinary stimuli, there is an exaggerated predominance 
m the animal of either excitation or inhibition, analagous respectively to 
the hysteria or neurasthenia observed in human patients Language nas 
treated in terms of higher order conditioning as a second signalling sys 
tern in which words bore the same relationship to conditioned stimuli as 
conditioned stimuli do to the U CS 

In spite of the elaborate nature of Pavlovian theory its fruitfulness m 
yielding experimental ideas, and its apparent relevance to important 
issues severe criticism was directed at the theory for a number of reasons. 

In summary, some of these are the following 
I Concepts of cortical physiology should be based upon more direct 
measures of cortical function Pavlov’s inhibition and excitation are 
purely inferential concepts denied from measures of overt raoicments or 
of amount of sain a secreted Likewise irradiation concentration, and iu 
duction are derived solely from the behaitoral observations of gcneraliza 
tion discrimination and after effect The concept of drainage is merely a 
figure of speech without any accepted neurophysiological basis Infcrvn 
till concepts are of course necessary m the systematization of the expert 
mental data, but little is gamed by asserting that such concepts deno e 
cortical processes unless specific aerification con be obtained lbiwi t li 
more direct observation of the cortex 
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9 Tlie temporal characteristic of irradiation, one of the most fund - 
mental points in Pavlov's systematization, does not rest upon adequate 
experimental verification. A careful analysis by Loueks of the original 
data of the experiments on irradiation indicates that there is no signifi- 
cant evidence for a sluggish spreading of an inhibitory disturbance trom 
one restricted region to surrounding areas” (1933, p. 44). The genera 
validity of any simple statement of irradiation for all regions of the 
cortex must be seriously doubted in view of findings by Dusser de Barenne 
and W. S. McCulloch (1938) that neural activity induced by application 
of stryc hnin e to certain points on the cortex shows almost no spread of 
activity to surrounding cortex, although in other areas there is irradia- 


tion to widely distant regions. This irradiation, moreover, is not sym- 
metrical. S tim ulation of a point in one area activates a second area, 
whereas the second area may not activate the first. A given point may 
augment the spontaneous activity of one area and depress that of a 


second. 

3. Assuming that Pavlov’s cortical mosaic is a matter of the extent to 
which different portions of the body are represented on the brain, the 
results of certain studies of generalization call this interpretation into 
serious question. Grant and Dittmer (1940a), for example, compared the 
generalization gradients for Galvanic Skin Responses conditioned to 
tactile OS’s on the back (small area of cortical representation) and on 
the hand (large area of cortical representation) and found the gradients 
to be essentially equal in slope. The irradiation hypothesis requires a 
steeper generalization gradient for the latter stimuli. Moreover, as Grant 
and Dittmer point out, Bass and Hull (1934) had obtained a gradient 
without inversions from shoulder to back to thigh to foot in spite of the 
fact that, in cortical representation, thigh, calf and foot are interpolated 
between the waist and the buttocks. 

4. The spatial character of Pavlov’s conception of cortical irradiation 


limits its general application. The assumption of a geometric projection of 
stimulus attributes onto the cortex does not seem applicable to generaliza- 
tion among stimuli differing in intensity or quality. In these cases the 
distance of cortical irradiation cannot determine the degree of generali- 
zation. The difficulty becomes clear in considering generalization in the 
intensity dimension, where the neural excitation may be mediated by 
identical fibers but at different frequency. All discrimination may per- 
haps depend on some sort of dynamic separation of areas of excitation 
as suggested in the Gestalt concept of isomorphism (Kohler, 1929) but 
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that there is a propagation of neural impulses by mean . „ 

° 6 y pavlov? a T if 1 0,1; ^ * P0SsibiIity remains t0 be demonstrated 
Pavlov s physiological conceptions are explicitly based unon the 

premise that conditioning is exclusively a cortical function. Experiments. 

level d Zr Str f^' kmera '' tlMt CODdit!oain S « possible at a subcortical 
level. Although the importance of the cortex in learning by higher mam- 
mals is unquestionable, any theory that neglects the functioning of lower 
levels of the nervous system must be inadequate. Pavlov’s judgment, that 
conditioning is impossible in the absence of the cerebral cortex, was based 
upon the failure of Zebony (192 9) in 1912 to establish conditioned re- 
sponses in dogs after surgical removal of the cerebral hemispheres. Later, 
Pol tyre v and Zeliony (1930) were able to establish motor conditioned' 
responses to visual and auditory stimuli in three decorticate dogs, and 
Lebedinskaia and Kosenthal (1935) were successful in conditioning a 
salivary response to a metronome. In all of the animals the operations 
left a small remnant of cortex at the base of the brain, and so great was 
the influence of Pavlov’s opinion that, in both studies, the conditioning 
was attributed to the residual cortical tissue Clear evidence of subcortical 
conditioning, however, has been presented by Ten Cate (1934c), who 
established a conditioned response to a bell in a totalling decorticate cat, 
and by Culler and filettler (1934), Girden, Mettler, Finch, and Culler 
(1936), Poltyrev (1936), Zeliony, and Kadykov (1938), who trained de- 
corticate dogs in conditioned responses to optic, cutaneous, and auditory 
stimuli based on unconditioned electric shock to the foreleg In some of 
these studies and in one by Broroiley (3948) the decorticate animal has 
shown to be capable of forming simple discriminations. 7 


Alternative Conceptions of the Place of Conditioning 
in Psychology 


These criticisms of Pavlov’s theory of cortical functioning seem to in- 
dicate that the conditioned response probably does not represent a direct 
' Although conditioning in decorticate animals seems well established, tbero is *nl) 
controversy as to whether conditioning is possible in the spinal carnal The ' P»»w *T 
of establishing CR’s in the isolated spinal cord was first suggested bj 
Ouller (1937) and Sburrager and Culler (1938) or flexion responses in spinal dog*. 

ms; W<r, mi; Xcii™, Dorn, 

and Deese, 194G; Piato and Bromi y» to onerative procedure* and ba»c concluded 
have usually attributed UCIlSr at least a response «h«h i* 

- = 3T "’- 
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reflection, of brain mechanisms. For this reason the importance of classical 
conditioning is to he sought in other connections. Psychological theonsts 
hare offered a variety of interpretations of this sort As has been stiesse 
before, conditioning won its acceptance in psychology by appealin t 
a set of influences which had dominated the Held for centuries. For this 
reason it is not surprising that the first evaluation of conditioning Vi as 

in terms of this background. _ ... 

1. Conditioning as a Substitute for Association. One such view was 
that conditioning was a substitute for association. S. Smith and Guthrie 
gave the initial impetus to this usage in their textbook (1921) . They used 
the term conditioning for a case in which the association occurs between 
stimuli and movements rather than between ideas. They extended the 
language of conditioning to the whole range of psychological phenomena. 
Conditioning, in their hands, became a general formula which provided 
a single principle of learning, analogous in importance to the primary 
law of association, contiguity. Guthrie’s theory still retains its heavy 
reliance upon the principle of contiguity. The connection to classical 
assoeiationistic psychology has, however, largely been lost, probably be- 
cause psychology, generally, is presently not much interested in the prob- 
lems central to classical psychology. 

2. The Conditioned Response as the Unit of Habit. Another early view 
of the conditioned response was that it is the unit of all learned activity 
or that complex habits are simply compounds of conditioned reflexes. 
Thus Pavlov states : “ It is obvious that the different kinds of habits based 
on training, education, and discipline of any sort are nothing but a long 
chain of conditioned reflexes.” (Pavlov, 1927, p. 395). This is the position 
which Watson promulgated, and it enjoyed a great vogue in America for 
some years. Prom the beginning, however, attacks on Watson’s statement 
of conditioning theory were numerous and vigorous. It is now clear on 
both theoretical and experimental grounds that simple chaining of con- 
ditioned responses will not predict the characteristics of complex habits 
The component responses are greatly altered by virtue of their combina- 
tion with other responses, and the influence of combination must be ex- 
perimentally determined. Watson’s simple conception of the conditioned 

31 aS u? e Tl ° f habit 18 110 longer lield b y an y°*e in the field of 
m D) although it occasionally is resurrected by critics of behaviorism 

^ Bergmann (19=6) says, “to scare little children in the existentialist 
Thus the early interpretations of the place of conditioning have disan- 

sense, representative of all learning Totoll response is, m some 

ing is a limited or special ^ ** 



CONDITIONING IN HISTORICAL PERSPECTIVE 39 

3 Conditioning Experiments as Representative of Other Forms of 
Learning If the eon<h tt „ nffig expenment proTes to « 

the essential characteristics of other learning, it should be a^eful 
' “ ’! Ueb t0 Sear f Sm hws o£ iearmn 8 It is obviously under better 

“ervdat We 1 ”™!! a 6 PraCt,CaI fut “ at ™ s ,a education and 

everyday life, and to the extent that it is a “pure ease" of learning, it 

should be a good source for the determination of the principles irhieh 
hold true for all learning Symonds (1927) presented an early exposition 
oi this possibility He listed 23 laws of conditioning and showed the 
analogies between them and learning in practical situations Such reason 
mg, however, shares the weakness of much argument by analogy, m that 
it calls attention to what may be merely superficial similarities, and it 
oversimplifies the relationship between conditioning and more complex 
learning 


A sounder view, which is a somewhat different version of this position, 
regards the conditioning situation as a source of deductive principles 
This is the position most prominently advocated by Hull, who used con 
ditioning principles to deduce such facts as the backward order of the 
elimination of blind alleys m maze learning and the excess of errors in 
the middle of a memorized senes of nonsense syllables In these applica 
tions, the deduced behavior is very unlike the simple conditioned response 
used in the deduction In the hands of Hull and others, the conditioning 
experiment has become the tool for harmonizing many facts already 
known about learning, and for predicting a variety of new facts 

This use of conditioning does not carry the implication that all habits 
are compounded of simple conditioned responses or that the principles 
discovered in conditioning experiments are directly applicable to complex 
habit situations The conditioning experiment, because of its simple and 
well controlled structure is a fruitful source of postulates from which 
deductions can be made concerning complex learning The following 
statement from Hull is a straightforward expression of this point of new 
It also presents a major issue on which Hull's position differed from that 


of Watson 


In order to correct a frequent misunderstanding, due presumably to the wide 
dissemination of the views of J B Batson, tho unter wishes to mate it «!«»* 
clear that neither here nor in any previous publication has he assumed that the 
more complex terms of behavior are sjnlhesmod tms micsss «h,cb |>Uy tho 
role of building blotto This may or may pot be (rue 1I» »«rlin B 
rather, that tie pnncplrs of acl.o» discovered in condemned reaction crpen 
meats are also operation in the higher behavioral processes The 
icctivo m tho expression, 'conditioned reStx pnnc.pl, V oooonlmsh refen. lo 
the loens of * scoter, of the principles rather than to their lorn, of oprra 
(Hull, 1933c, p 227a) 



CONDITIONING AND LEARNING 

4 Conditioning os a Subordinate and Restricted Form of Fining 
Psychologists who object to the loregomg point of tow do so for one 
„£ two reasons: Either they are opposed to all association or conditioning 
theories of learning, or they believe that the conditioned response repre- 
sents a limited form of learning. The Gestalt psychologists belong to the 
former group. They see conditioning as epitomizing the atomistic-mech- 
anistic approach to science and they object to it on that ground. 

The argument of the other group is that there is more than one kind of 
learning iTolman, 1919) and that conditioning is only one of these forms. 
Moreover, the common further assumption is that different laws govern 
the progress of the different forms of learning. The most common state- 
ment of this idea maintains that (a) it is useful to discriminate between 
classical and instrumental conditioning, and that (Z>) classical condition- 
ing is contiguity learning and instrumental conditioning is governed by 
the law of effect. Whether this view is correct or not is a theoretical ques- 
tion upon mliich agreement has not been reached in psychology. The next 
two chapters will present some of the basic facts associated with classical 
and instrumental conditioning, and these hopefully, will provide a basis 
for adopting one point of view or the other on this question. 


SUMMARY 

The immediate reaction of psychology to Pavlov’s demonstration of 
the conditionability of the salivary reflex mas one of enthusiastic accept- 
ance. The conditioning method was hailed as providing psychology with 
a method as scientific and objective as the methods of physiology. And 
the concepts of conditioning mere offered as explanatory devices of mide 
applicability. It mas proposed that conditioning theory mould improve 
our understanding in such diverse fields as social behavior, psychopath- 
ology , and the learning or school subjects. Prom our present vantage 
pomt such enthusiasm seems unmarranted. Much of mhat me have at- 
tempted to do in this chapter has been to try to explain this uncritical 
initial reaction to the facts and theories of conditioning. 


Philosophical Background 

To a very important extent, it is possible to understand the easy ac- 

bacWomd Tlf ° ntaS f r “' ip ' es “ terms 01 Psychology's historical 
background. Tie general approach which dominated psychological 

thought at the time the conditioning experiment was introduced wi a 

dmcct contmuahon of a pattern o£ thinking established by the BrSst 
empirical philosophers. Tmo asnpnk r»f + j-n- v Lue -°ritisn 

importance. One was the idea oftalvsi^s atyTi I? ° £ 
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c t~ and tiiat fte *• -»«• otaiudin. 

is to isdate these elements so as to reveal the true nature of the events ,n 

"t:; reflex was acceptei - tie « 

nf if™” 3 d,r “* T t l lbUtlQn ° f tte BntKl1 E “P>"^ts was the concept 
of association On the basis of their analyses of the mind these phiioso 
phers had proposed that mental phenomena are composed of simple ideas 
bound together by associations The early structural psychologists had 
accepted this interpretation and had set out to discover the elements of 
human experience by the method of introspection The elements they dis 
covered were unanalyzable sensory units welded together to produce con 
sciousness, they claimed, by the laws of association Although Pavlomn 
theory differed considerably from this interpretation at the level of 
factual content, the basic idea was the same Jn the case of the condi 
tioned reflex the bond between conditioned stimulus and conditioned re 
sponse was very similar to the familiar idea of association 


Scientific Background 


While psychology's philosophical history had prepared it to accept the 
style of thought which was implicit in the concept of the conditioned 
reflex, other developments had prepared it even more directly to accept 
the subject matter For one thing, psychology had always maintained a 
close tie with biological science Early research on the sensory processes 
had led to an interest in physiology and to a tendency to place great faith 
m physiological explanation Conditioning was introduced by a world 
famous physiologist and described as a physiological method This account 
of the conditioning method made it a part of a respected approach to 
psychological problems, and did so at a time when traditional psj chology 


was in serious trouble 

With the increasing use of lower animals m psychological research the 
older introspective psychology winch mainly dealt with sensor}’ attn 
butes and the association of ideas was finding itself poorly equipped to 
deal with certain experimental observations Among the most difficult for 
a psychology designed to bundle the rational characteristics of the mind 
was the fact that the behavior of lower animals was often irrational emo- 
tional, and almost reflexly automatic Faced with the problem of describ- 
ing such actions some psj chologists hod already turned aua} from the 
older conceptions and had begun to develop new ones suggested by the 
work of Darwin There was an emphasis upon the adjustue character of 
behavior, epitomized by the concept of habit. The translation of habitual 
behai lor into the concepts of conditioned reflexes was an easy matter 

aC F,nSs*m all biological science as well as m psychology there was a 
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. , ; n the direction o£ objectivity. This trend constituted a 

powerful contemporary mtrospectionism, which seemed 

overly subjective. 0 Conditioning was accepted, in part, because it seeme 

entirely free of such subjectivism. 


Russian Background 

The Russian phvsioloeist Pavlov first encountered the phenomena of 
conditioned reflexes incidentally in his studies of the gastric secretions 
associated with digestion. Recognizing the unportance of what he had 
observed, he turned his considerable scientific talents to the direct in- 
vestigation of these “psychic secretions.” After initial crude and rela- 
tively uncontrolled experiments, Pavlov developed his now famous condi- 
tioning methods and carried them out under conditions of rigid control. 
With these methods Pavlov discovered all of the basic phenomena of con- 
ditioning and developed an elaborate quasi-neurological theory to account 
for them. 

Pavlov’s theory of conditioning treated behavioral phenomena in the 
purely materialistic tradition of physiology begun by Sechenov. The basic 
concepts were those of excitation and inhibition, conceived as cortical 
events. The interactions of these two processes provided an explanation 
for the various facts of conditioning. In spite of this strength, Pavlov’s 
theory is objectionable for a number of reasons, of which the most im- 
portant are these: (1) We now know that conditioning is possible in the 
absence of the cerebral cortex; so more than cortical function must be 
involved. (2) Certain facts of conditioning do not fit the theory. Many 
of these facts involved the phenomenon of stimulus generalization. (3) 
What passes for physiology in Pavlov’s theory is nothing more than the 
translation of behavioral facts into neurophysiological language. 


The History of Conditioning in America 

A contemporary of Pavlov’s, Bechterev, was mainly responsible for 
bringing the details of the conditioning experiment to the attention of the 
■ft extern world, because bis books were translated into known languages 
much earlier than Pavlov’s. By about 1915, the conditioning experiment 
was reasonably familiar to American psychology, and there had been a 
few pioneer experiments. From the beginning, however, it was not the 

, * ut the concepts of conditioning, which had the important impact 
upon American psychology. * 

The most tapcrtani application of conditioned-reflex concepts las al- 

TrVTv AT “ deTe, ° P a 8 ™ Cral theor y learning. Initial 
01 tlus sort > adopting the mode of thought set down hy the earlv 
sensory psychology, assumed that all habits are synthesized of unitary 
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conditioned reflex elements Tins view is no ionger popular Contempo 
vary theorists who hypothesize that the conditioned reflex is basie to learn- 
ing hold that, although the laws of conditioning may apply to all learning, 
this does not mean that every learned act is made up of a chain of reflexes 
These theorists have attempted to show that the laws of conditioning may 
be used as a set of postulates from which the phenomena of more complex 
learning may he deduced 

Other theorists disagree with this approach, either because they think 
that conditioning and other forms of learning are quite separate proc- 
esses, or because they regard this heavy stress on deduction as inappro- 
priate at psychology’s present stage of development Theorists who take 
either of these positions are apt to agree on the point that we need much 
more factual information than we now have before attempting theoretical 
'organization of the psychology of learning 
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Classical and Instrumental 
Conditioning Experiments 

The basic distinction between classical and instrumental conditioning 
procedures is in terms oflthe consequences of the conditioned response.) 
In classical conditioning, t he sequence of events is independe nt of the 
-Subiects^behaviQr. In instrumental conditioning, by contrast, rewards 
and punishments ar e made to occur as a consequence of the learnePsTe^ 
_ sjonse or j ailure to respog[Th^33tereiiee between these two procedures 
did not seem important to the earliest investigators 0 f conditioning and 
the two procedures were used quite indiscriminately. In some experi- 
ments with shock, for example, the shock was unavoidable and occurred 

r:r ~ ™ 

methods has become a matter of very great in“" 

tssi- * - - — - 

CLASSICAL CONDITIONING 

ditionefsXa^ioMnTogs 1 The1^S^ ti0,,il1 ? \ ^ ^ ° f COn ' 

be presented by describing a tv™ l ature s of the procedure may 
(Anrep, 1920) : In orde^o ^1^7“ ? Pa7 ‘° 7 ' S lab ” a 4 
salivary response, the dog was first snh' T? 8 ° £ the ma Sutude of the 
which the duct of the parotid -land w« • , . eEted t0 a minor operation in 
through an opening on the ouMe J tl i T rtad f> ‘he saliva flowed 

firnity cemented over the optbg to t u«1h T“ waa 

measured with an accuracy of one tenth f l ™’ wMch couW ^ 
dog was trained to stand quietly hi am 7 7 SUltable d ™ces. The 
Which was insulated againlt any ^ “ * tab,a *» a room 

U). The experimenter occupied S' (Figure 

through a small window and present « “ g r00m ' Serving the dog 
devices. A tuning fork was 

^ ° V 8 Seeonds ^ter the begin- 
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FIGURE 3.2. Pavlov’s arrangement for the study of salivary conditioning 
From Yerkes and Morguhs (2909) 


ning of this stimulus, a plate containing a small measured quantity of 
dry powdered food was moved within reach of the dog’s mouth No sali- 
% ation was evoked by the tone, but during the eattng there was a copious 
flow of saliva Combinations of the tone and food were presented three 


times during a daily session, separated by intervals of from 5 to 35 min- 
utes The strength of the conditioned response was determined by pre 
senting the tone alone for 30 seconds and measuring the amount of saliva- 
tion After ten double stimulations there was a slight conditioned saliva 
tion, and after thirty combinations the tone evoked a salivation of si vty 
drops On the early tests the conditioned salivation did not occur until 
the tone had sounded for 18 seconds, on later tests the salivation began 
after only 1 or 2 seconds Both measures indicate an increase m the 


strength of the conditioned response with training 

Another example of classical conditioning which employs^ motor 
rather than a secretory response, may be selected from Liddell s (1934) 
studies on sheep A metronome was set beating once per second for o 
seconds and then a shock was delivered from an tndnctonum through 
electrodes attached to the left foreleg of the animal The metronome had 
no apparent effect upon the sheep’s behavior, whereas the shock evoked a 
definite brief flexion of the limb, accompanied by changes m respiration 
and m electnenJ res, stance o£ the shin (The Guhomc Skm ^po^ . 
GSR) After four presentations of the metronome and shook in comb, nn 
tion, the conditioned stimulus caused a definite change «> bmt J 1,1 » 
s m resistance, and on the sixth trial there nos a slight iiio.cmc . of the 
“^.th trial the conditioned leg flexion nas fully established 

and xvitli further traminjj nas quite “‘’s^i'lhe ™nd,M°ned 

of the eleventh trial I art , UlustraJ id 4 fofloxx.np the onset of 

P— - «•' 

“ a motor response, , the «*■ 
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Left forleg 
Respiration 
Metronome (CS) 
Shock (UCS) 
Time (A sec) 
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FIGURE 3.2. Conditioned responses in the sheep. The diagram represents a 
tracing from a kymograph record of the eleventh presentation of a metronome 
(conditioned stimulus, CS) combined with a shock (unconditioned stimulus, 
UCS) delivered to the left foreleg. The metronome heat once a second for 
5 seconds prior to the shock. Both the conditioned leg movements and respira- 
tory changes began while the metronome was sounding, anticipating the 
shock. These conditioned responses (CR) may be distinguished from the 
unconditioned responses (UCR) following shock. After Liddell (1934, p. 
268). 


tioning experiment. Such studies, most often with human subjects, usu- 
ally employ an airpuff as a UCS to elicit a reflex blink, although loud 
noises, mild shocks, and taps on the cheek have been used for this purpose. 
The response has sometimes been recorded photographically (Dodge, 
1926). More recently, electronic devices have been used to record the 
blink. The CS may be any detectable stimulation. Mild visual and audi- 
tory stimuli have been most commonly used. 


Basic Concepts 

The essential features of the conditioning experiment are the follow- 
ing : 

1. Unconditioned stimulus (UCS)— a stimulus which, at the outset of 
an experiment, evokes a regular and measurable response. 

2. Unconditioned response (UCR)— the regular and measurable re 
sponse to the unconditioned stimulus just referred to. It is worth men 
tiomng specifically that the unconditioned response is not necessarily ai 
unlearned or reflex response to the UCS. In many cases, such as salivatio: 
to rood placed m the mouth, it is-, but in other cases, such as salivation £ 
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UCS^LTot ^ “ tie d,St “ Ce ’ ** i '-'- a.so used as a 

UcVt’t! Mned tT to (OB)-**""*-** (.) doesuot esohe the 
UCR, m this respect being neutral, and (4) „ paired with the IJCS tor 
experimental purposes Again one point is worth mentioning often the 
,, d e ' oke an observable response, it only the reaction invoked m 
paying attention to the CS, which Pavlov called an orientation reflex 
Ulhat is it reflex, or investigatory reflex In some experiments, such as eye- 
lid conditioning with a visual CS, the response to the CS may even be 
oi the same general kind as that to the UCS, in this example, a blink In 
neither case, however, is the response to the CS exactly like that to the 
UCS The investigatory reflex is always qualitatively different, ami re- 
sponses of the other kind differ from the UCR quantitatively, being of 
different latency and magnitude More of this later 
4 Conditioned response (CR) — sometimes, as a result of the pairing 
of CS and CCS, a response resembling tbe UCR follows the CS, either on 
test trials with the UCS omitted or m anticipation of the UCS Such a 
response is the conditioned response Note especially that the CIt mereJ> 
resembles the UCR This is another point to which we shall return later 


Classical Conditioning Procedures 
The ordering and temporal spacing of stimuli in the classical condition 
ing experiment have given rise to a number of terms which describe 
different forms of the experiment In the typical conditioning expert 
ment, the CS begins a brief time before the onset of the UCS It may 
either overlap the UCS in time or terminate before the appearance of 
the UCS The time between CS and UCS may be short or long and so on 
The technical terminology of conditioning describes such relationships 
Figure 3 3 is a schematic representation of some of the possible relation 
ships of CS to the UCS la this figure, time is represented on the top 
line The lower lines in the figure indicate stimulus onset and cessation, 
respectively, by upward and downward jogs The second line represents 
the occurrence of the UCS which is the same for all of the following 
examples The third line shows the arrangement for a simultaneous con 
ditiomng procedure in which the CS and UCS come on together The 
next four lines illustrate delayed conditionin'; procedures in which the 
CS appears prior to the UCS and lasts at least until the latter stimulus 
appears It, then, may go off as the CS cornea on (line I), go off « hen tie 
CS goes off (line 5), or persist beyond the termination of the ( b (line 0) 

As line 7 shows, the CS may be either stimulus onset or e.ssation Tins is 
hue of the other forms of conditioning too, although no cxunplr. are 
presented in Figure 3 3 Lmc 6 shows n trace eonMiomng “ 

which the CS comes on briefly and goes off ag on before the UCs Line 
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FIGURE 3.3. Some possible relations between CS and UCS in classical condi- 
tioning. Lines 1 and 2 represent, respectively, the passage of time and the 
onset (upward excursion of the line) and cessation (downward excursion) 
of the UCS. Line 3 illustrates the placement of the CS for strictly simultane- 
ous conditioning. Lines 4, 5, 6, and 7 are examples of delayed conditioning 
procedures. Line 8 presents the procedure for trace conditioning. Line 9 
illustrates backward conditioning. 


illustrates backward conditioning in which the CS follows the UCS. Con- 
ditioning is difficult and perhaps impossible to obtain with this method. 

One final conditioning procedure, not illustrated in Figure 3.3, is 
temporal conditioning in which a time interval functions as a CS. An 
unconditioned stimulus, such as food, is presented at regular intervals 
of time. If it is now omitted, a conditioned response may occur at approxi- 
mately the usual interval. Pavlov obtained conditioning in dogs, using 
intervals as long as 30 minutes. 


Organisms Which Have Been Conditioned 

One of the earliest questions to arise in the history of classical condi- 
tioning concerned the generality of the process. This question breaks 
down into several more specific ones involving (1) the range of re- 
sponses which can be conditioned, (2) the stimuli to which conditioning 
is possible, (3) the phylogenetic limits within which conditioning occurs, 
and (4) within a given species, the effect of age upon ease of conditioning. 
Tins section is concerned with the last two of these questions. 
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Th» ptajuman (flatworm), which has bilateral symmetry ani a cere 
bral ganglion, seems to be the simplest organism winch is certainly 
capable of forming classical conditioned responses, although its perform- 
ance is not impressive Thompson and McConnell (1855) conditioned 
planana in a light shock situation The CS was provided by taming on 
two 100 watt lights The UCS was a 28 milba mpere shock passed through 
the water m a plastic trough used for the apparatus Two different condi 
turned responses were observed, a turning of the head to one side or the 
other and a longitudinal contraction of the entire body In 150 trials the 
probability of occurrence of one or the other of these responses increased 
from just under 30 per cent to just over 40 per cent For this, and all 
higher phylogenetic lex- els, there seems to be no doubt that conditioning 
can occur 

There haie been far fewer investigations of the ontogeny of condi 
tionabihty than there have been of the phylogenetic range within which 
conditioning can occur Such studies as have been done, however, show 
that conditioning is possible at a very early age, almost certainty prior to 
birth E L Hunt (1949), employing chick embryos as subjects, paired 
a bell with electric shock and obtained conditioning of a gross bodily 
movement on the fifteenth day of incubation In some cases the response 
was retained until after hatching Spelt (1938, 1948) apparently obtained 
conditioned responses from human fetuses 6 1/2 to 8 1/2 months of age 
He used a very loud noise, which produced an unconditioned startle like 
response, as the UCS and a tactile vibration of the mother’s abdomen as 
the CS Movement of the fetus was measured by tambours attached to 
the mother’s abdomen After pairings of CS and UCS, movement re- 
sponses occurred to the CS alone 

Dorothy Marquis (1931) conditioned the sucking reactions of new- 
born infants to the sound of a buzzer 1 She also showed that it is possible 
to alter the feeding schedule of the neonate to a degree by manipulating 
the amount of time between nursings Morgan and Morgan (1944) sug- 
gest the conditioned eyebhnk as a possibly useful developmental index 
They gave 42 infants of different ages 100 conditioning trials Their 
procedure consisted of bringing the hand with a baby sjnnge (CS) near 
the eye of the infant and squeezing it to produce a puff of air (UCS) 
which made the baby blink To test for the existence of conditioning, the 
experimenter brought the sjrmge Dear the baby’s e> e, but did not squeeze 
it A blink to the syringe under these circumstances is a CJ1 In tests ol 
children from 15 to 121 days of age, Morgan and Morgan found that no 
child 53 days of age or jounger conditioned, whereas only one infant CG 
days of age or older failed to condition 
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Responses Used as Unconditioned Responses 

U\er the years, conditioning studio >,* i, 
ety or responses. Table 1 lists sometf the les ““ 0arried 0nt a 
together with the unconditioned stjm^ WUch >> a ™ been use, 

hem. At least in terms of procedure Y ™ be “ to elie 
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it - 1 e most basic questions to be asked - 3 6Se CaSes ^ tlle same 
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ztT d r r°“ re - 

flexiou of a limb, „ ^ ’SZST. 

uZ fc'^v m ™ arc as wtamd, tinned stimuli. S iL 

I f ^ of spinal animals ls difficult, and perhaps impos- 

Tla L^ M i .T Ple , ““ are Mtremel J' difficult to condition 
The abdominal, patellar, plantar, and pupillary reflexes fall in tins cate- 
gory. When conditioning does occur is may be that voluntary facilitation 
is an important factor (ScUosberg, 1939). The reason for the difficulty 
encountered in conditioning may relate to the fact that these reflexes are 
very slightly involved in the processes of motivation and reward. 2 


TABLE I 

and the Stimuli Used to Elicit Them 

Stimuli 

Dry food, acid 
Light 

Electric shock 

Change in illumination, shock 
Food 

Shock, thermal stimuli 
Morphine 

Injection of toxin and antigen 
Increased water intake 
Electric shock 
Patellar blow 
Shock, sound, air puff 
BolaJjnn 

Electric shock 

Electric shock 

Electric shock 

Food 

Shock 

Various 

Various 


Unconditioned Responses 

Response* 

Salivation 

Blocking of EEG Alpha Rhythm 
Change in skm resistance (GSR) 

Pupillary reflex 
Gaatro intestinal secretions 
Vasomotor reactions 
Nausea, vomiting, and so on 
Immunity reactions 
Diuresis 
Flexion reflex 
Knee jerk 
Eyelid reflex 
Eye movements 
Change in respiration 
Change m pitch of voice 
Withdrawal movements 
Mouth opening, swallowing 
Locomotion 
Instructed responses 
Previously conditioned (higher 
order) responses 

Conditioned Stimuli 

Although unconditioned stimuli are chosen in reference to the response 
to be elicited, any environmental change to which the organism u sonsi- 

■la lie laboratory, the popular, reflex is real,!, dirttal outer .nib a 
illumination or by administration of so elcctne .tori MUm s h 1 he Jira.oo cltUto* 
no this boars is not perfect, success in conditioning the pupillary redes has been notably 
bette^in eapenments emjiloy.nsr *«« <>»v‘ « <«■»» Wely that 
accompaniments of tbe shook may have f” 2 ” A$g? SLl!, 

tte pupillary seta -a. been by 

nSm°hX™?£Z Keti.arev.ky flUBJ, Sf.unos ««i Piter IWU"* 
Crs.Jucrk and MeCra^ie ( s ,„£ ,, (IMO), strebta and llen.ba. (1WI), *■« 
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tive may serve as a conditioned stimulus. Visual stimuli have included 
lights of various colors, papers, geometrical forms, rotating objects. Audi- 
tory stimuli have included pure tones, horns, buzzers, bubbling water, 
metronomes. Various thermal, tactual, olfactory, and proprioceptive stim- 
uli have been used. Pavlov tended to prefer continuing stimuli, such as 
metronomes, electric fans, rotating disks. Experimenters using reflexes 
in striate muscles have tended to prefer stimuli with sharp onset such 
as flashes of light, clicks, contact stimuli. The effectiveness of some stimuli 
may depend on their sudden onset, and others may depend for their 
effectiveness on summation. It is known that a change in the intensity of 
a stimulus or the termination of a stimulus may serve as well as' the 
appearance of a stimulus. 

Internal stimuli are not as frequently employed in conditioning erneri- 
ments as external stimuli because they are not so susceptible to° prLTe 

SrSSTST V* *“• ‘here have been 

studies in which the CS was direct stimulation of the brain (n 61 and 

described by EL-an in'lds TOh iJ t h ^r 3 lUustrath ’ e sample is 

Intestinal loops were isolated C ° nditioni ^ : 

provided for the introduction of liquids «o sera afcS’sl T* 

tine. For one dog the CS was a 9 * \ f s LS s) mto the mtes- 

for the other it was a 5.5 per cent soluti™ f ‘‘l “ ° f hydrochlori ' ! 

UCS was a shoeh administered to ftetf, v f g ‘"'T P ea< * case the 
The animal receiving the hydroehlorie^cid CSfo" ^ 15 SeCOnds later ’ 
m just sixteen trials. The animal r • • ° S formed a stable motor CR 

— CSandasiine^r^^^ 
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ement or incomplete record- 
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ln £f of the details of the reaction Time , , , , 

fw Tr, , °" dCn ' 1935 ' ““W 19 ») . tat It seems , m ,e celt’ 
P,1 ■ ' J 9Ua f tat, ' e reeords tohU weal differences Smularly 
v s records of salivation seldom provide an indication of tlie fem 

o n 1 ?o ‘T“ atl ° n Wi ‘ en SUCh records have be en provided (Kon 
orski, 1M8) differences between CR and UCR are easily detectable This 
evidence that the CR and UCR are different suggests the importance of 
attempting to determine the precise nature of the relationship between 
them The views held most commonly have been that the CR is either a 
fractional component of the UCR, or that it is a preparation for the 
occurrence of the UCS 


Tlie CR Viewed as a Fractional Component Response 
In many experiments, the conditioned response appears to he weaker 
than the unconditioned response or to represent only a component part 
of it The conditioned salivary response for example is often of smaller 
magnitude than the unconditioned response Moreover where the un 
conditioned response to food consists of salivation chewing and swallow 
ing, the conditioned response may involve only one or two of these com 
ponents A dog which has received shock on the leg together with a buzzer 
may give a conditioned withdrawal response without the vocalization 
that was a part of the reaction to shod 
Although it is doubtful that the description of the CR as a fractional 
component of the UCR provides a complete explanation of the former 
response, one point about such responses is very important namely tint 
the fractional component can occur in the absence of the unconditioned 
stimulus and at points remote from it in tune or space Hull made con 
siderable use of this characteristic of the CR m a series of papers dealing 
with complex behavior, and many of the same explanations appeared in 
his last book (Hull, 1952) In these treatments Hull referred to the com 
plete unconditioned response as a goal response (Ro) and to the com 
ponenfc as a fractional anticipatory goal response (r G ) IVc shall meet the 
concept of r c , its proprioceptive stimulus sa the combination 

r > sc, at a number of later points in the book kmong the phe 

nomen a to which it has been applied are latent learning frustration 
place learning delay of reinforcement problem solving and conflict be* 
fn.\ior (Hull, 3952) In each of these cases the fractional anticipate^* 

BM 1 responses lias been mode to sene much tin same function ns the 
idea of expectancy does in the cogmiiic theories. I robably no s. vlc 
development m cither camp has done quite so much to nvoncle < 1 . 
cnees between the two theories (Behan, 1533, Seward, Mo&O 
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The CR Viewed as a Preparatory Response 

Other writers have taken the position that the function of the condi- 
tioned response is to prepare the organism for the occurrence of the un- 
conditioned stimulus. In such discussions it is often noted that the CR 
may not resemble the UCR at all. It is as if the subject “expects” the 
forthco min g stimulation and prepares to receive it. The preparatory 
character of some conditioned behavior in the Pavlovian situation has 
been described by Zener (1937). Salivary responses were conditioned to 
a bed which preceded feeding in the usual manner. The total behavior 
was recorded by motion pictures, and the differences between conditioned 
and unconditioned behavior were summarized as follows: 


Except for the component of salivary secretion the conditioned and unconditioned 
behavior is not identical, (a) During most of the time in which the bell is rein- 
forced by the presence of food, chewing generally occurs with the head raised out 
of the foodpan but not directed either at the bell or into the foodpan, or at any 
definite environmental object. Yet this posture practically never, even chewing 
only occasionally, occurs to the conditioned stimulus alone. Despite Pavlov’s 
assertions, the dog does not appear to be eating an imaginary food. (b)Nor is the 
behavior that does appear an arrested or partial unconditioned reaction consist- 
ing of those response elements not conflicting with other actions. It is a different 
reaction, anthropomorphieally deseribable as a looking for, expecting, the fall of 
food with a readiness to perform the eating behavior which will occur when the 
oo alls. The eUector pattern is not identical with the unconditioned, (c) Move- 
ments frequently occur which do not appear as part of the unconditioned response 

p° 393 ) 1 01 ° f Stamphlg ’ P^ting. (Zener, 1937, 
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The Spec.fica.ioa of d.e CondxUoned Response 
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FIGURE 3 5 Conditioned and unconditioned responses in the same subjects 
on adjacent trials The upper line in each panel represents the onset of the 
light employed as a CS (upward jog) and the UCS which was an oirjwll 
(downward jog) half a second later The CR is distinguishable from the 
UCR in terms of latency The CR anticipates the onset of the UCS Unpulw 
lished records of the writer 

latently and characteristically different form In the actual conditioning 
situation the conditioned eyebhnk is recognized as such because it ap 
pears m anticipation of the UCS This criterion alone however is not 
sufficient to identity the CR because there are several other responses 
which may also occur in the interval between CS and UCS and be 
confused with a true CR These other responses include redex blinks 
to the CS ( alpha responses), beta responses ( see below) voluntary re- 
sponses, and random blinks. With the possible exception of the rau 
dorn blink, these responses differ from CR s m latency , form, or both 
Table 2 presents typical latency ranges from various responses which 
may occur in an eyelid conditioning situation bigurc dG illustrates 
some of the ones which do not appear w hi 0 ure 3 5 The latency figures 
in Table 2 assume an interval between CS and UCS of about £ acctmh 
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FIGURE 3.6. Varieties of response which occur in the eyelid conditioning 
situation. Note the short latency and distinctive form of the typical voluntary 
response presented in the left-hand panel in the middle row. The right-hand 
response in the same row has the form of a voluntary response hut the latency 
of a true CR. Experimenters must decide whether to count these as CR’s or 
as voluntary responses. The record in the lower right-hand panel illustrates 
the problem encountered when anticipatory responses of small amplitude 
appear. Whether or not this is a CR depends upon the experimenter’s criteria. 
Unpublished records of the writer. 

The latency characteristics of both CR’s and voluntary responses change 
with changes in the interstimulus interval. 

The beta response mentioned in Table 2 was discovered by Grant and 
his co-workers (Grant, 194:5 ; Grant and Norris, 1947). It is a reflex blink 
to light which is sensitized by dark adaptation. Figure 3.7 presents the 
results of one study in the form of a frequency distribution, which shows 
the latency characteristics of the alpha, beta, and conditioned responses. 
With subjects run in the light (lower graph) the responses in the beta 


TABLE 2 

1 ARIETIES OF RESPONSES IN EYELID CONDITIONING 

Experiments and Characteristic Latencies 


Response 

Latency Range in liilliseconds 

Reflex blink to light (Alpha response) 

Beta response 

Conditioned response 

Voluntary response 

Random blink 

Unconditioned response 

50 — 110 See Rig. 3.6 

5-0 — 240 See Pigs. 3.6, 3.7 

250—500 See Pigs. 3.5, 3.6, 3.7 

200—500 See Pig. 3.6 

See Pig. 3.6 

50 — 100 msec, after UCS 

See Pigs. 3.5, 3.6 
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FIGURE 3 7 Latency distributions of reactions evoked by the CS in an ejelid 
conditioning experiment under conditions of dark adaptation and light adap- 
tation The Beta responses are unconditioned reflexes to light vtlucli are 
easily confused with the true CR The small distribution of responses with 
latencies in the range from about 60 to about 120 milliseconds are tmcondi 
{toned reflexes to light (Alpha responses) In order to eliminate the con 
laminating effects of Beta responses experimenter* usually run their subjects 
under conditions of light adaptation and count as CK's only those responses 
with latencies greater than 6ome specified minimal value. Data from Grant 
and Norris (1947) 

range tend to disappear Those rcmninmg ire probablj main); random 
blinks 

The importance of an exact specification of the nature of the CR may 
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Beta response 
Conditioned response 
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Random blink 
Unconditioned response 
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FIGURE 3.7. Latency distributions of reactions evoked by the CS in on cjclid 
conditioning experiment under conditions of dark odaptation and light adap- 
tation. The Beta responses are unconditioned reflexes to light whirls are 
easily confused with the true CR. The tma 11 distribution of responses with 
latencies in the range from about 60 to about 120 milliseconds arc uncondi- 
tioned reflexes to light (Alpha responses). In order to eliminate the con- 
taminating effects of Beta responses experimenters usually run their subject* 
under conditions of light adaptation and count as CR's only those response* 
with latencies greater than some specified minimal value. Data from Grant 
and Norris (1917). 

raugc tend to disappear Those renn tiling arc probably mainly random 
blinks 

The importance of an exact specification of the nature of the CR may 
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VARIETIES OF EYELID RESPONSE 


FIGURE 3.6. Varieties of response which occur in the eyelid conditioning 
situation. Note the short latency and distinctive form of the typical voluntary 
response presented in the left-hand panel in the middle row. The right-hand 
response in the same row has the form of a voluntary response but the latency 
of a true CR. Experimenters must decide whether to count these as CR’s or 
as voluntary responses. The record in the lower right-hand panel illustrates 
the problem encountered when anticipatory responses of small amplitude 
appear. Whether or not this is a CR depends upon the experimenter’s criteria. 
Unpublished records of the writer. 


The latency characteristics of both CR’s and voluntary responses change 
with changes in the interstimulus interval. 

The beta response mentioned in Table 2 was discovered by Grant and 
his co-workers (Grant, 1945 ; Grant and Norris, 1947). It is a reflex blink 
to light which is sensitized by dark adaptation. Figure 3.7 presents the 
results of one study in the form of a frequency distribution, which shows 
the latency characteristics of the alpha, beta, and conditioned responses. 
With subjects run in the light (lower graph) the responses in the beta 


TABLE 2 


Varieties of Responses in Eyelid Conditioning 
Experiments and Characteristic Latencies 


Besponse 


Latency Range in Milliseconds 


Reflex blink to light (Alpha response) 

Beta response 

Conditioned response 

Voluntary response 

Random blink 

Unconditioned response 


50—110 

120—240 

250—500 

200—500 


50—100 


See Pig. 3.6 
See Pigs. 3.6, 3.7 
See Pigs. 3.5, 3.6, 3.7 
See Pig. 3.6 
See Pig. 3.6 
msec, after UCS 

See Figs. 3.5, 3.6 
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FIGURE 3 7 Latency distributions of read ions evoked by the CS tn an eyelid 
conditioning experiment under conditions of dark adaptation and light adap 
taboo The Beta responses are unconditioned reflexes to light which are 
easily confused with the true CR The small distribution of re<ponsci with 
latencies in the range from about 60 to about 120 milliseconds are unrondi 
tioned reflexes to light (Alpha responses) In order to eliminate the con 
laminating effects of Beta responses experimenters usually run their subjects 
under conditions of light adaptation and count as CR*s only those responses 
with latencies greater than some specified minimal »alue Data from Crant 
and Noms (1947). 

range tend to disappear Those remaining are probably mainly random 
blinks 

The importance of an exact speeiflc.it ton of the nature of the CIl may 
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be illustrated by considering the effect which a failure to distinguish 
among voluntary, beta, conditioned, and unconditioned responses has 
probably had upon the history of eyelid conditioning. Early studies were 
almost always run with the subjects in the dark, a procedure necessitated 
by the photographic method of recording responses. This means that 
many of the responses counted as CR’s in these studies were beta re- 
sponses and that the progress of conditioning was not accurately repre- 
sented. Voluntary responses have introduced a somewhat different set of 
complications, which may be illustrated by the early eyelid conditioning 
studies on rats (Hughes and Schlosberg, 1938), dogs (Hilgard and Mar- 
quis, 1935), monkeys (Hilgard and Marquis, 1935, 1936), and infants 
(Morgan and Morgan, 1944) . In each of these studies, the unconditioned 
response was lid closure to a puff of air directed at the eye. Records from 
dog, monkey, and man are reproduced in Figure 3.8. The gross similarity 


DOG 


”71 

Light 



Air puff 


\Aaa/ 



MAN 



FIGURE 3.8 Tracings of records obtained from dog, monkey, and man in the 
eyelid conditioning situation. The original records were photographs ob- 
tamed with Dodge’s photochronographic procedure. In such records eyelid 

hS 3 £ Sgs 
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of the results m the different studies must not be permitted to obscure 
the differences A careful examination of the conditioning records shows 
that the form of the response is different for the different species and 
that the latency is somewhat shorter m the lower species than in man 
Indeed the responses for dog and monkey are like the voluntary responses 
of human subjects, and probably are voluntary Informal and uapub 
lished observation of the conditioned eyelid response in infants (Kimble) 
leaves no doubt at all that what they develop is a voluntary conditioned 
response rather than the automatic reflex like CB 


A final example in the same general connection concerns the value of 
requiring the subject to make a “mediating” response, such as a key 
press, to the CS Studies of this sort (Miller and Cole, 1936, Beck, 1939) 
have routinely demonstrated much better conditioning and greater resist 
ance to extinction with than without the added response Careful studies 
of records obtained with this procedure, however, show that the act of 
pressing the key produces a strong tendency to blink simultaneously, and 
that these unconditioned blinks are indistinguishable from low amplitude 
CR’s (Ost, 1960) Thus the reasonable hypothesis that a mediating motor 
act aids eyelid conditioning remains to be validated 


Further Complexities 

As we have just seen the conditioning process presents many prob 
lems even at the relatively simple level of the identification of the condi 
tioned reflex Further consideration of the behavioral changes which 
occur in conditioning reveals that this is only the beginning of the mat 
ter The conditioning procedure sometimes leads to other modifications 
which require separate specification and treatment, these include adapta 
tion, a conditioned suppression of the UCB, pseudoconditiomug and re 
flex sensitization 

1 Adaptation Presentation of a noxious UCS alone prior to a condi 
Uomng procedure sometimes leads to a diminution of response strength 
known as adaptation Systematic studies of this phenomenon have not 
been numerous, but the results have been quite consistent MacDonald 
(1946) conditioned the eyelid reflex and finger retraction m human sub 
jects with and without a senes of 50 prior adaptation trials with the 
UCS, air puff, or shock for the two different procedures. By comparison 
with control groups which received no adaptation trials she found that 
the effect of the adaptation trials was to reduce the number of conditioned 
responses from 27 7 to 6 0 in the case of the cjcblmk, and from 27 1 to 
10 0 m the finger retraction experiment Tailor (1056) studied theeffee 
tiveness of different levels of UCS, using three adaptation groups md 
a control group in an evchd conditioning experiment Expressed in term* 
of millimeters of mcmirj (a measure of the pn-ssurc behind the puff) 
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the TJCS intensities during 50 adaptation trials were 80, 30, 15 and^O 

( i nrrami For these croups respectively the percentages 
(control group), hor tnese v * Kimble and 

conditioning trials were 9.14, 1 A2U, Ib.db, ana 

0956) present data from a similar study in a form winch shows 
2f Seet dtkg the course of conditioning. They gave 20 adaptation 
trials with the UCS to their experimental group and none to a conti o 
oroup in an eyelid conditioning experiment. The results appear in Figure 
3.9. Presumably the relatively small amount of adaptation is a function 
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FIGURE 3.9. Conditioning curves following with and without previous adap- 
tation trials with the UCS. The CS in this experiment was a light, the UCS 
an airpuff (Kimble and Dufort, 1956). 


of the limited number of adaptation trials. The effect of different num- 
bers of such trials seems not to have been investigated. 3 

2. Pseudoconditioning. Curiously, the procedures for producing adap- 
tation sometimes have exactly the opposite effect: Instead of a decrease 


^. h ® stll ^ e ? “team sec tion have all concerned adaptation of the response to 
the PCS, which has been the matter of greatest interest in this area. Adaptation does 
h0W . e7er ‘ McAllister (1953) found nearly complete adaptation of a 
Wr^TtriSs. a * 33 a CS in eyelid conditioning over a period of one 
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in ease of conditioning, they produce responses like conditioned responses 
on the very first presentation of the conditioned stimulus Such reactions 
are called pseudoconditioning, because they resemble true CR's hut do 
not depend upon double stimulation After a reflex to light bad been 
adapted out in fish, so that the light no longer evoked a response, the 
fish were shocked repeatedly, no illumination change accompanying the 
shock Following this treatment, both light and a vibratory stimulus pro 
duccd reactions which might have been interpreted as conditioned if the 
light and vibration had been combined with the shock (R R Sears, 1934) 
Similarly, monkeys were frightened several times by a powder flash (or 
a “snake” blowout) Following this senes of experiences a bell was 
sounded, and fnght responses were evoked which the bell bad not preu 
ously elicited (Grether, 1938) Grant (1943a, 1943b) has demonstrated 
pseudoconditioning of the eyelid reflex Harlow (1939) used a carefully 
determined threshold shock as a CS and a strong shock as a UCS m a 
Study with goldfish, in which he compared forward conditioning back 
ward conditioning, and pseudoconditioning Trials with the strong shock 
alone produced stable pseudoconditioning very quickly Similarly 10, 20, 
or 30 shock trials produced increasing amounts of psuedoconditiomng 
in cats, with the function essentially asymptotic after 20 trials (Harlow 
and Toltzien, 1940) In this latter study, tests were postponed for 5 mm 
utes, 3 hours, or 24 hours after the shock trials Pseudoconditioning 
occurred to about the same degree in each of these conditions The im 
portance of this fact is that it seems to indicate that an explanation in 
terms of generalized excitement is unsatisfactory Such cTCJlcmcnt (fear 
anxiety) would presumably subside in 24 hours An alternative possibil 
lty seems more likely that conditioning of responses to shock to tho 
general situation meditates the pseudoconditioned response 
The most extensive study involving pseudoconditioning was that of 
Harris (1941) with finger withdrawal He compared forward condition 
ing backward conditioning and pseudoconditioning with a random con 
ditiomng procedure in which CS and UCS were presented alone The CS 
was a loud tone which lasted 4 75 seconds the UCb was a 25 second 
shock In the forward conditioning situation the shock followed the tone 
by 4 75 seconds All groups received a total of 80 shocks The measure 
of the effectiveness of the different procedures was percentage of condi 
tioned responses on a series of 10 trials without shock The results on this 
measure appear m Table 3 These results actuallj show a higher propor 
tioo of “CU s” after the pseudoconditioning procedure than anj other 
Harris points out that the responses to tone in all conditions except the 
forward conditioning one are probably pseudoconditioned responses 
Kimble and his associates (Jvimblc Mann ami Dufort, 19 >j, Kimble 
and Dufort, 39o6 Dufort md Kimble 19o3) fcavc human subjects a 
series of conditioning trials and then omitted the CS on a scries of trials 




FIGURE 3.10. Graph showing the effect of interpolated trials with the UCS 
upon performance. Data from Kimble, Mann, and Dufort (1955). 


in eyelid conditioning. The procedure of omitting the CS is, of course, a 
pseudoeonditioning one. Figure 3.10 presents a set of typical results. 
Quite clearly, the trials with the TICS alone facilitated conditioning as 
would he expected from other evidence on pseudoconditioning. These 

TABLE 3 

Percentage of Responses to Tone Following 
Various Training Procedures 

(From Harris, 1941) 


Condition Extinction Trials 



1 

2 

3-4 

5-6 

7-8 

9-10 

Forward conditioning 

90.0 

70.0 

60.0 

35.0 

30.0 

35.0 

Backward conditioning 

84.6 

46.1 

30.7 

23.0 

26.8 

23.0 

Random presentation 

44.4 

33.3 

22.2 

16.6 

22.2 

22.2 

Pseudoeonditioning 

100.0 

80.0 

30.0 

10.0 

20.0 

10.0 


studies make the additional point, however, that the effect of the UCS 
trials may he exactly the same as those in which CS and UCS are paired. 
The implication of this is that pseuxloconditioning may he a part of all 
conditioning in which a noxious stimulus is employed d 


Recently this finding has teen questioned. Experiments attempting to replicate the 
reauits ui Figure 3.10 have been unsuccessful (Goodrich, Ross, and Wagner, 1957). 

No satmraetory explanation lor thia disparity has yet been advanced 
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A further application of the pseudoconditiomng procedure is to be 
found in the restoration of extinguished responses With the omission of 
the UCS, the conditioned reaction gradually subsides or extinguishes 
Following such extinction the conditioned response is completely restored 
(reconditioned) by a few pairings of the CS and UCS It may also be re 
stored m another way, by presenting the UCS alone, without the CS, for 
several times In this case, again, unpaired applications of the UCS have 
the effect of producing an increment in the strength of the response, after 
the manner of pseudoconditioning 

The mechanism by which pseudoconditioning and related phenomena 
occur (if, indeed , there is only one mechanism ) is not known Sears (1334) 
and Harlow (1939) have employed a variant of Ukhtomsh's (1926) 
concept of dominance for tins purpose According to this principle, any 
reaction which has been repeatedly elicited by a strong stimulus may 
become dominant over reactions, so that it is ehcitable by a much wider 
range of stimuli than is normally the case Grether (1938) and Harlow 
and Tolfczien (1940) have suggested that pseudoconditiomng occurs be 
cause of the development of an attitude of expectancy It is characteristic 
of all of these explanations that they are performance, or nonassociatue, 
interpretations No learning process is involved 

Wickens and Wickens (1942), on the other hand, have interpreted 
pseudoconditioning as a case of true conditioning They believe that dur 
mg training with the noxious stimulus, the organism acquires a response 
which is transferred to the neutral stimulus on test trials because the two 
stimuli are, in some sense, similar Both, for example may come on sud 
denly To test this hypothesis, Wickens and Wickens trained two different 
groups of rats to escape from a shock which came on suddenly for one 
group and gradually for the other Subsequently, each group was sub 
divided and tested with a light which came on suddenly or gradually 
Ultimately the four groups consisted of two m which the onsets of shock 
and light were similar (both sudden or both gradual) and two groups in 
which the onsets were different (one being sudden the other gradual) 

On the tests with the light 15 of 29 animals m the two groups where the 
stimuli were similar showed pseudoconditiomng by performing the re> 
sponse originally learned to escape the shock In the case of the animals 
tested with dissimilar stimuli, however, only 3 of 18 produced a pseudo* 
conditioned response These results support the idea that pscudocondi 
tiomng may actual!} be a form of ordinary learning 5 
Unfortunately the amount of experimentation on pseudoconditiomng 
is \ery small It seems quite likely that paucity of cudence is directly 
traceable to the connotations of the prefix pseudo, and that research on 


* Tha experiments ot Grant and Dittmcr (10*0) rfiowlag RtwalmUoa of Ik* 
pseudocondition cil reflex and Switzer (1033) demonjtr&Ung dninUUUo* * 

similar interpretation. 
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an important problem has been inhibited on grounds that are merely 

3 Inhibition of the UCR by Double Stimulation. Although the normal 
effect of pairing the CS and UCS is to produce conditioning, the pro- 
cedure is known to have two others: 1) a suppression of the response to 
the UCS and 2) sensitization of the response to the CS. This section is 
concerned with the first of these influences; the next section is concerned 
with the second. Figure 3.11 presents a set of records from an eyelid 
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FIGURE 3.11. A selection of response records showing the inhibition of the 
unconditioned reflex on trials ivith the CS and recovery on trials without the 
CS. Note that no CR’s occur on these trials (Kimble and Ost, 1960). 

conditioning study which show this phenomenon quite clearly. In this 
study the CS was a light, followed in .5 second by the UCS, an air puff. 
During paired presentations of the two stimuli there is what appears to 
be an adaptation of the UCR. Removing the CS, however, restores the full 
magnitude of the UCR, indicating that the response diminution resulted, 
not from adaptation, in the sense of reflex fatigue, but from an inhibitory 
process under the control of the CS. The nature of this inhibition is not 
completely understood, having, when observed, been attributed to the 
interaction of reflexes (Hilgard, 1933b), personality factors (Pfaffman 

™c^ SberSl 1936) ’ and Pavlovian inhibition of delay (Kimble and 
ust, lybl). 


Sensitization 

The augmentation of the response to the conditioned stimulus through 



CLASSICAL AND INSTRUMENTAL EXPERIMENTS 65 

fi U n? rt x 0ndltl0mng ™ detecfced m conditioning tlie knee jerk by Wendt 
(1930), in conditioning the eyelid reflex by Hilgard (1931) and Bern 
stem (1934), and in conditioning a startle response in rats by Prosser 
and Hunter (1936) In the Prosser and Hunter experiment, auditory 
stimuli just too weak to elicit startle responses were paired with an elec 
trie shock This procedure increased excitability to the point where the 
startle reaction was eh cited by the previously mefiectn e sound It seems 
unlikely that this was a true conditioned response because its latency was 
the same as the latency exhibited with more intense sounds In a /ater 
experiment, Hunter (1937) showed that true conditioned startle reac 
tions differed in latency and other characteristics from the unconditioned 
reaction 

Subsequent work on sensitization has most often involved the condi 
tioned eyelid reflex Hilgard and Biel (1937) reported that they were 
unable to obtain the effect However, two later studies were successful 
In one of these, Weber and "Wendt (1942) obtained alpha conditioning 
as an incidental result in a study concerned with a comparison of differ 
ent unconditioned stimuli More substantial evidence for the reality of 
the process comes from a study of Grant and J K Adams (1944) de 
signed specifically to demonstrate the phenomenon In tins study the 
percentages of alpha responses were compared under two different condi 
tions One was a regular conditioning procedure in which a CS (light) 
and UCS (air puff) were paired for 50 trials The other was an adapta 
tion procedure consisting of 50 presentations of the CS alone In the con 
ditiomng procedure, alpha responses occurred on about 45 per cent of tha 
trials The comparable percentage under the adaptation procedure was 
about 16 per cent The frequency of the occurrence of the alpha response 
increased rapidly during the first few reinforced trials and then showed 
a slight tendency to decline 

INSTRUMENTAL CONDITIONED RESPONSES 
The classical Pavlovian training procedures described in the previous 
section do not exhaust the methods of establishing simple learned re 
sponses We turn now to a presentation of sc\cral other methods which 
have also come to be called conditioning These training methods domed 
from the work of Bel litercj-XmiEThowidih e differ from the classical 
Patlovum procedures m that the subject s belm w n instrinncntni to the 
production of reward *oT~ ~s(o.i_<hnee of p umshffl en t lienee the name 
instr umental conditioning 

Twofold classifications of learning corresponding roughly ia the dis- 
tinction between classical and instrumental conditioning ha\c been pro- 
posed by other writer* Table 4 presents the tcnmnologj suggested b> 
seier »! of them 
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These classifications are purely operational ones and describe tramm 0 
methods rather than underlying processes. Where the distinction is made, 
\however, the reason for it often lies in a belief that the two kinds of learn- 
ing obey different laws. Of these the two commonest are : (l)_that_classieal 
Conditioning is basically a process of association by con tiguity, whereas 
In strumental conditioning is under the cont rol of thejaw oCfrect^and/or^ 
(2 ) tha t classical conditioning operates mainly on responses of the auto- 


TABLE 4 


Twofold Classifications of Learning Proposed by Different Authors 


Author (s) 

Term for Classical 
Conditioning 

Term for Instrumental 
Conditioning 

Thorndike (1911) 

Associative shifting 

Trial-and-error learning 

Miller and. Konorski 
(1928) ; Konorski 
and. Miller (1937a, 

Type I 

Type II 

lyo/D) 

Skinner (1937) 

Type S, or Eespondent 
Conditioning 

Type E, or Operant 

Sehlosberg (1937) 

Success learning 

Hilgard and Marquis 
(1940) 

Mowrer (1947) 

Classical conditioning 

Instrumental conditioning 

Conditioning 

Problem solving 


■nomlc nervous-system, w hereas instrumental conditioning is limited to 
nonautonomic r eactions. A detailed discussion of such interpretations is 
reserved for the next chapter. 




Procedural Variations 

Distinctions among a variety of instrumental conditioning procedures 
to he presented in this section turn upon what is basically Thorndike’s 
distinction between satisfiers and annoyers, or in contemporary terminol- 
ogy, positive and negative reinforcers-. “By a satisfying state of affairs 
[positive reinforcer] is meant one which the animal does nothing to avoid, 
often doing such things as attain and preserve it. By a discomforting or 
annoying state of affairs [negative reinforcer] is meant one which the 
animal commonly avoids and abandons.” (Thorndike, 1911, p. 245). This 
statement shows that the term reinforcement and its various derivatives 
may have a purely operational and essentially factual meaning. It also 
leads to the identification, on logical grounds, of four possible instru- 
mental conditioning procedures, in which the response under observation 
produces or avoids a positive or negative reinforcer. This classification of 
instrumental training procedures was proposed by Konorski (1948) and 
overlaps m some degree that of Hilgard and Marquis (1940) . 
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Reward Training 

Instrumental reward training has been studied extensively for many 
years m many different animals and with various responses It is the same 
as that described by Lloyd Morgan in 1894 and designated a few years 
later by Thorndike (1898) as trial error, and accidental success i- 
Thorndike carried out a series of experiments in which eats were trained 
to ese&pe from problem bores to secure food outside The boxes were con 
structed with sides of vertical slats so that the food was visible to the cat 
A door in the box opened as soon as the cat performed a certain move- 
ment such as pulling a string hanging from the top or pressing a latch 
'/A hungry cat when first placed m such a box shows a continued but lari 
able activity reaching between the slats toward the food scratching at 
the sides moving all about the box In the course of this activity the cat 
eventually hits the release mechanism and immediately secures the food 
The first successful response appears to be largely a matter of chance On 
successive tnals the cat s activity becomes concentrated in the region of 
the release mechanism and other activities gradually drop out until 
eventually the animal performs the correct act as soon as it is placed in 
the box 

In more recent years psychologists have introduced reward training 
procedures designed to further the investigation of less complicated be 
havior The most common of these June been the T maze the straight run 
way (Graham and Gagn<* 1940) and the Skinner box (Figure 3 12) m 



FIGURE 3 12 The key pecking apparatus for pigeon*. In the £ 

ment the pigeon is reinforced lor j cckmg the key when it >• d 
Piecing Men or * wonocl romalor helwcen lhe .* } " tU 

makes »l P<mihle «ry die mieniuy and hue o f d e ght la * j 

translucent key The .ketch .» adapted from one pubh.heJ by Cuttm* 
Ivali.h (1958) 
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which an animal, usually a rat or a pigeon, learns to operate a device of 
some sort to obtain food. In eaet of them the sub 3 eet receives food for 
performing the correct response. 

The conception of instrumental act must not be understood too nar- 
rowly. In the illustrations which have been given, the subject pro- 
duces the reward in some mechanical fashion, either by bringing itself 
into the presence of the reward through its own movements, or by bring- 
ing the reward to itself through some release mechanism. However, the 
act which precedes the reward need not be mechanically related to 
the delivery of the reward, but instead may be purely arbitrary. For 
example, in one experiment Thorndike taught the cat to lick itself in order 
to be released from the cage. Training the dog to “shake hands” by re- 
warding is another illustration of an arbitrary act of this sort. The only 
essential aspect of instrumental training is that the reward follows the 
response in some systematic fashion. 

Most reward-training experiments proceed in a series of discrete trials, 
with a signal, such as raising the door in a runway, marking the begin- 
ning of a trial. Skinner’s procedures, however, do not employ trials, but 
rather a free-responding situation. When the rat or pigeon makes the 
appropriate response it is immediately free to make the same response 
again. Because of this training method, the progress of learning is 
measured in terms of rate of responding, that is, the number of responses 
per unit of time. 


Avoidance Training 

In the usual form of the avoidance-training experiment, the subject, 
by responding to a signal, prevents the appearance of a noxious stimulus. 
The reference experiment is the classical one by Bekhterev on conditioned- 
withdrawal responses of hand or foot, in which the responding member 
rested on the electrode, and electric shock could be avoided by the per- 
formance of the conditioned response. If the electrodes are attached to 
the subject and the shock is delivered on every trial, the experiment is 
changed to a classical conditioning one. The majority of recent avoidance- 
learning experiments have been with lower animals. Some of the common 
responses investigated in such procedures are lever pressing, wheel turn- 
ing, and hurdle jumping. 

As is true of reward training, most of the studies of avoidance training 
use tnal-by -trial procedures. Sidman, however, has carried out a series 
oi mvestigations of avoidance training in a free-responding situation 
(Sidman 19o3a, 19o3b). In Sidman’s work, rats are used as subjects in 
a modified Skinner situation. At a fixed interval, such as every 20 seconds 
the rat is shocked unless it presses a bar in the interval. H it does tt 
shock is postponed and, therefore, avoided for a certain amount of W 
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say 30 seconds Under these conditions rats develop high rates of respond* 
xng Note especially that no discrete exteroceptive -warning signal is used 
in this method 

Another procedure, employing no definite warning signal, closely re 
lated to avoidance training, and seemingly a subvariety of it, is escape 
training In this method the instrumental response turns off a noxious 
stimulus This type of training procedure has not been widely used m 
laboratory conditioning experiments, although numerous examples are 
found in the learning of animals in their natural environments An ex- 
periment reported by Mowrer (1940) may serve as the paradigm of 
instrumental escape training A rat was placed m a box with a metal 
grille floor through winch an electric shock could be administered The 
shock was gradually increased during 2 25 minutes from zero to a stable 
intense value At one end of the box was a pedal arrangement which 
would turn ofii the shock whenever pushed The shock remained off as 
long as the pedal was depressed, but when the pedal was released the 
shock would automatically start to build up again 
Rats m this situation began to show agitated behavior about 60 seconds 
after the beginning of the shock As the shock became more intense they 
engaged in vigorous undirected activity characterized by jumping, run- 
ning, squealing, biting at the grille, and random thrashing about In the 
course of this activity the rat always chanced to hit the pedal and thus 
terminated the shock The pedal response on the first trial occurred onlj 
after from 3 to 6 minutes of activity, but by the tenth trial the rats 
pressed the pedal as soon as they began to feel the shock The obvious 
difference between this and the more usual avoidance training situation is 
that the warning signal and the unconditioned noxious stimulus are in 
the same modality In this example, weak values of shock serve as the 
signal to which the rat can respond and avoid the stronger shocks to 
come In other escape training procedures, the shock is turned on full 
force at the beginning of a trial, and the subject must learn to respond 
quickly to it (Sheffield and Temnicr, 1930) 

Omission Training* 

Reward and avoidance training arc the forms of instrumental condi- 
tioning most frequently used m the laboratory Probably tins is because 
they represent a parallel to the majority of real life learning situations 
in which the organism performs m order to receive some positive rein 
foreement or to escape from or avoid some negative reinforcement At the 
same time, somewhat less frequent cvcrjdij situations, and logic, both 
suggest two other instrumental training procedures. In the first of these 

• I* D Sheffield (personal communication) suggested iMs nnn-o Ur ti« J wrduffc 
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a specified response would lead to the nonoccurrence of a positive rein- 
forcer; in the second, it would lead to the occurrence of a negative rein- 
forcer. The first of these situations is roughly the same as that in which 
the parent withholds privileges for undesirable behavior. We have de- 
cided to call it omission training. The second procedure is essentially the 
punishment situation, and we have called it punishment training. The 
only extended discussion of these methods available in English seems to 
be tha t of Konorski (1948, Chap. 12). 

Omission tr ainin g is a procedure in which a positive reinforeer occurs 
if the org anis m fails to make some particular response ; that is, if the 
specified response occurs, reinforcement is omitted. In this respect it 
differs from the extinction procedure in which reinforcement never oc- 
curs. Konorski (1948) describes this method and its outcome in a series 
of experiments carried out on dogs. A given stimulus, such as a metro- 
nome, is first paired with food and a classical CR formed to it. Then, 
occasionally, the metronome is set in motion, and at the same time, the 
dog’s leg is passively flexed. On such trials food is omitted; on other trials 



FIGURE 3.13. Diagram of one of the experimental arrangements nsed by 
Konorski and S. Miller in Pavlov’s laboratory, a— system to measure saliva- 
tion; b system to measure flexion of the dog’s leg; c — system to produce 
passive flexion. From Razran (1939d). 


the metronome continues to be reinforced with food. Under this training 
procedure (see Figure 3.13) Konorski reports that : 


We shall soon find that the raising of the leg becomes a conditioned inhibitor 
i.e., -when applied concurrently with the metronome it inhib its the secretion of 
saliva to this stimulus. But in addition, more or less at the same tW the animal 
begms to resist the passive flexion of the leg by actively extending it After a 
time the annual pexuorms an active movement of extension to th°e metronome 
itseli. . . . Finally the movement of extension becomes so strong that we are 
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194s1> a ^ 22 tT the anmal mt ° ** ^ by ltS extended “ (Konorsln, 


The important thing about this observation is that it shows that nourern 
forcemeat does not lead merely to a failure of the CR to develop but 
rather to the strengthening of an antagonistic response 
In passing, it should be mentioned that Konorski’s work represents 
one of the very few research programs m which an investigator has 
studied classical and instrumental responses established simultaneously 
m the same situation The importance of this is that it provides a way of 
studying the possibility, held by many theorists, that classical condition- 
ing provides the basis upon which instrumentally conditioned responses 
are developed If this were the case one would expect the two forms of 
conditioning to respond to various experimental treatments w roughly 
parallel ways Such results as can be gleaned from Konorski ’s report sup- 
port such a view 


Punishment Training 

This procedure consists m making the occurrence of some noxious 
stimulus, such as electric shock or the introduction of a mild acid solution 
into the mouth, contingent upon the occurrence of a specified response 
It should be noted carefully that this procedure is different from (and 
essentially the opposite of) avoidance training, in which the painful 
stimulus occurs only if the to be learned response fails to occur More- 
over, in the avoidance training situation, the noxious stimulus, if it occurs 
at all, is administered before the response In punishment training the 
noxious stimulus occurs after it. Konorski reports one such method in 
which the procedure parallels that described above as omission training 
In these experiments, mechanical flexion of a hmb, in the presence of a 
CS, is reinforced (punished) with acid Under these circumstances, the 
dog again learns an antagonistic extensor response It is a matter of some 
interest that responses leading to the production of a negative rcinforccr 
are influenced in exactly the same way as those which lead to the omission 
of a positive one, and, conversely, responses leading to the avoidance of 
a negative remforcer behave like those leading to the production of a 
positive one 

The more common experiments on punishment differ from IConorski ’s 
m the behavior investigated The usual experiment is concerned with the 
use of punishment as an adjunct to an extinction procedure The mam 
interest is in the speed with which some response is abandoned, rather 
tlnn m the development of antagonistic behavior For this reason the 
American literature contains verj few, if any, experimental demonstra- 
tions which parallel Konorski ’s. 
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Organisms W hidb. Have Been Conditioned 

As with, classical conditioning, a question of great interest in the early 
history of instrumental conditioning involved the phylogenetic range 
over which the process occurs. Efforts to condition one-celled organisms 
have usually employed instrumental conditioning procedures and have 
produced contradictory results. Beatrice Gelber 7 attempted to obtain 
conditioning in the paramecium in the following manner: She dipped a 
platinum wire, which had been swabbed with a fluid containing bacteiia 
upon which the paramecia feed, into a culture of paramecia for a series 
of 40 trials. She found that the number of paramecia congregating near 
the wire increased steadily as a result of such training, and attributed 
this outcome to conditioning. The CS, in such an interpretation, is the 
stimulation provided by the wire; the DCS is food, and both the CR and 
IJCRr are instrumental responses which bring the animals near to the wire. 
The evidence for conditioning is the increasing number of subjects col- 
lecting in the region of the platinum wire. Both Jensen (1957) and Katz 


and Deterline (1958), however, have argued that Gelber ’s procedure 
may have increased the bacterial concentration in the vicinity of the wire 
and that the paramecia may only have been gathering in baeteria-rich 
areas. These investigators have also presented data to show that such an 
interpretation is possible. Control colonies into which food was intro- 
duced without the training series behaved in ways indistinguishable from 
those subjected to the conditioning procedure. Such evidence leads to a 
rejection of Gelber ’s conclusion that her procedure produces condition- 
ing. It is possible, of course, that some other method might. 

There is, in fact, some evidence that even paramecia can learn; but 
rather than consisting of the formation of new association, it involves the 
inhibition of unnecessary responses. French (1940), for example, ob- 
served the behavior of paramecia in learning to escape from a glass tube 
.6 mm. in diameter to get to a culture medium. Half of his subjects 
showed a significant improvement in performance in terms of time re- 
quired to reach the culture medium. Moreover, French noted specifically 
that this improvement resulted from the elimination of extraneous re- 
sponses which occurred early in the test series. 


Most if not all of the successful experiments employing one-celled 
animals have produced this form of learning, rather than the formation 
of a new association (Warden, Jenkins, and Warner, 1940). Even at 
slightly higher phylogenetic levels learning seems to be of the same sort. 
F eure and Walton (1907) reported an experiment on the sea-anemone 
m which pieces of filter paper were placed on certain tentacles at ^4-hour 
intervals. At first the tentacles grasped the filter paper and carried it to 
the mouth where it was swallowed and, later, rejected. After two to five 
T Gelber (1952, 1954, 1956a, 1956b), Gelber and Rasch (1956). 
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trials, however, the paper was rejected by the animals and the tentacles 
so trained failed to grasp the paper at all Again note that the response 
is an inhibitory one and that the conditioning procedure is instrumental 
As with classical conditioning, studies employing animals phylogenet- 
ically higher than the very simplest have usually been successful De 
scnption of instrumental learning in a great range of species may be 
found in any booh on comparative psychology 


The Nature of Conditioned Instrumental Responses 

Instrumentally conditioned reactions are usually specified m terms of 
their consequences All reactions which depress the bar m the Skinner 
box, turn the wheel a certain distance in the wheel turning apparatus, or 
close the electrical contact m the key pecking situation qualify as rc 
sponses, no matter how much they may differ in detail At least up to the 
present time, the researcher m instrumental conditioning has not been 
forced to concern himself with the description of the exact form of 
the CR in the way which has been found to be necessary in eyelid 
conditioning 

Since m all types of instrumental learning the response which 13 
learned must occur before it can be reinforced the response is obviously 
already in the behavior repertory of the subject and the learning process 
consists only in strengthening the response so that it occurs promptly and 
with dependable frequency m the presence of the conditioned stimulus 
One concept appropriate to the description of instrumental conditioning 
m such terms is Hull’s (1934a) concept of habit family hierarchy The 
organism, at the outset of training m the instrumental conditioning situa 
tion, has at its disposal a variety of responses {habit family) which vary 
in strength, or form a hierarchy Since it is a lei ruing situation, the 
dominant response is, by definition, incorrect and instrumental condi 
tioning consists of a reordering of the strengths of the v arious responses 
m the habit family hierarchy 

An obvious question concerns flic origin of the responses in the habit 
family hierarchy, since they are released independently of identifiable 
external stimuli Tor this reason Skinner (1938) refers to them as 
emitted acts, or as operants, to distinguish them from elicited reactions or 
res2)omIc»ts, evoked by recognized stimuli They are not, however, en 
tircly random as the trial and error theorists havo sometimes implied 
Tlio cats m Thorndike's puzzle box spent much time on the first trial 
reaching through the slats toward the food and the rats in Skinners 
situation explored the corners and projecting portions of the box. In 
some casts, these reactions seem to be native responses to stumihtion In 
Others they arc reactions to dmc or ru.uk from previous training 
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Responses Determined by tbe Drive 
Often the problem situation is characterized by the presence of a drive 
which increases the restless behavior. Since drives have been associated m 
the animal’s past with certain acts leading to reward, the responses likely 
to appear in a new experimental situation may include those which have 
occurred previously while the particular drive was active. Several experi- 
ments have demonstrated that animals can discriminate among their own 
drives, that a drive may be a conditioned stimulus evoking specific be- 
havior (Hull, 1933; Leeper, 1935). 


Generalized Responses to External Stimuli 

When an organism is in a novel situation, it reacts to features of the 
environment bearing some marks of familiarity. In conditioning terms 
this is an example of stimulus generalization; that is, previously condi- 
tioned responses are evoked by stimuli similar to the original learning 
situation. A rat which has secured food by digging in one box with a 
sawdust floor is more likely to begin digging when in a new box with a 
sawdust floor than is an animal without this previous experience. 


Special Training 

The appropriate response in instrumental learning may be encouraged 
in many artificial ways, depending upon the ingenuity of the experi- 
menter or animal trainer. The response may be suggested by passive move- 
ment. This is common practice in animal training, as in teaching a dog 
to “shake hands.’’ Or the leg movement which is rewarded may be orig- 
inally produced by shock to the foot. Lever pressing is sometimes en- 
hanced by placing a bit of food on the lever to direct the animal’s explora- 
tory activity to that region. String-pulling behavior is facilitated by 
smearing food on the string, jiggling the string, and so on (T. L. Mc- 
Culloch, 1934). The experiment proper consists in the strengthening of 
the response after it has once appeared. This is an important difference 
from Pavlov’s experiment. The difference is illustrated by the question: 
How many trials are needed before the first conditioned response ap- 
pears? This may be asked with respect to Pavlov’s experiment but is 
meaningless with respect to instrumental training in which the condi- 
tioned response must always occur before the first trial is completed. 


SUMMARY 


It is possible to distinguish between two forms of conditioning experi- 
ment, called classical and instrumental conditioning, by considering 
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Whether reinforcement is contingent upon the occurrence of the condi- 
tioned response In classical conditioning the unconditioned stimulus 
occurs whether or not a conditioned response occum In instrumental 
conditioning, by contrast, the conditioned response must occur if reward 
is to be obtained or punishment avoided 


Classical Conditioning 

The reference experiment in classical conditioning is the one with 
dogs, in which a neutral stimulus, such as the sound of a tuning fork 
was presented a few seconds before food After several such pairings, the 
dog began to salivate to the sound of the tuning fork In such experi- 
ments, the neutral stimulus (tuning fork) is the conditioned stimulus 
(OS) The originally effective stimulus (food) is the unconditioned stim- 
ulus (UCS) The original response (salnation) to the UCS is the uncon- 
ditioned response (UCR) And the new response (also salivation) to the 
CS is the conditioned response (CR) 

Varieties of classical conditioning procedure are defined in terms of the 
relationship between CS and UCS Those in which the CS precedes the 
UCS are called forward conditioning procedures If the UCS precedes 
the CS, the experiment involves backward conditioning, a method which 
is very ineffectiv e in the production of conditioned responses Thus most 
experiments employ the forward conditioning arrangement Within the 
category, forward conditioning, there are two subvarieties, delayed condi 
tiomng and trace conditioning In delayed conditioning the CS conics on 
before the UCS and continues at least until the onset of the UCS Iu trace 
conditioning the CS comes on only momentarily, some time prior to the 
UCS Conditioned responses, m trace conditioning, occur in the absence 
of the CS, perhaps to a “memory” trace of it A final classical condition 
mg procedure, temporal conditioning, involves the presentation of the 
UCS on a fixed time schedule, such as every 30 seconds In these expen 
meats there is no CS, unless the time interval is considered to be one 
The variety of responses, organisms, and stimuli successfully used in 
classical conditioning studies has been vciy great Such learning oppar 
cntly can occur prior to birth and in a range of organisms extending 
from at least the level of the flatwonn to man Stimuli employed have in 
eluded exteroceptive stimulation in most, if not all modalities as well as 
interoceptive stimuli applied to the internal orpins and directly to the 
nervous system Tho variety of responses studied in what have been 
called classical conditioning experiments is also very gnat We should 
reserve judgment, however, as to whether the procedure in these vxperi- 
ments have all been truly classical conditioning 
Although the conditioned and unconditioned response are usually 
similar m classical conditioning, it is a mistake to think of them o» 



exactly tlie same, or to imagine that the conditioned response is simply 
the TJCR transferred to the CS. There are always differences m at least 
the quantitative characteristics of the reaction. Because of this fact vari- 
ous views of the nature of the conditioned response have been set forth. 
These include the proposals that the CR is (a) a fractional component of 
the UCR and (Z>) a preparation for the occurrence of the UCR. 

Particularly in eyelid conditioning, in which the blink reflex is condi- 
tioned, much work has been done on the form of the responses which 
occur in the conditioning experiment and on the ways in which the re- 
sponse can be altered through means which are not essentially condition- 
ing. Among the different responses observed in such an experiment there 
are, in addition to the UCR and CR, unconditioned reflexes elicited by the 
CS, “beta” responses to light sensitized by dark adaptation, random 
blinks, and voluntary responses. These must be identified and excluded 
from the conditioning measure. Other processes which take place in the 
course of conditioning also complicate things. These include (a) adapta- 
tion of the response to the UCS, a phenomenon occurring at the receptor 
or reflex level, (Z>) a stimulus-controlled inhibition which resembles 
adaptation but is not, (e) pseudoconditioning, in which applications of 
the UCS alone produce responses like conditioned responses, and (cZ) 
sensitization, or alpha conditioning, which results in an augmentation of 
the unconditioned response to the CS. It is quite apparent that the seem- 
ingly simple conditioned reflex experiment actually contains features of 
great complexity. 


Instrumental Conditioning 

The instrumental conditioning procedure has been known to animal 
trainers for centuries and has been used in the laboratory since the 
1890’s. In this procedure what the animal does in the situation deter- 
mines what happens to it; technically, there is a response-reward con- 
tingency. In general, the arrangement is that some specific response leads 
to the presentation or omission of reward or punishment. Instrumental 
conditioning comprises four different procedures which we have called 
reward training, avoidance training, omission training, and punishment 
training. Of these the first two are most commonly used in the laboratory 
1. Reward Training . In this procedure, the performance of some re- 
sponse leads to a goal object for which the subject is motivated. With 
lover animals the responses required have involved running mazes press- 
ing panels and levers, (in the case of birds) pecking at illuminated disks 
pulling strings, and jumping across gaps in a runway. 

2. Avoidance Training. In this procedure the correct response, if per- 
formed promptly after a warning signal, prevents the occurrence of some 
noxious stimulus such as electric shock, so that the subject avoids it. In a 
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subvariety of avoidance training:, called escape training, the warning 
signal is omitted and the subject must respond quickly to the presenta* 
tion of the painful stimulus to avoid its continuation 

3 Omission Training In a few experiments, the procedure has been 
to omit reward when a particular response occurs Under these conditions 
the subject develops a tendency which is antagonistic to the response 
leading to the omission of reward Unfortunately there haie been very 
few experiments on this kind of learning 

4 Punishment Training In another form of experiment which has not 
been studied much, the occurrence of some specified response leads to the 
application of some unpleasant stimulus As in omission training the 
subject develops an active antagonism to making the response in question 

As with classical conditioning, instrumental learning occurs in a wide 
range of species In fact, one celled animals seem to be capable of a form 
of instrumental learning by which they acquire an inhibition of unneces- 
sary movements The range of responses and stimuli employed in studies 
of instrumental learning is also at least as great in classical conditioning 

In instrumental learning the response to be conditioned must occur 
before it can be rewarded or punished Therefore this response must have 
been in the animals' behavioral repertory prior to the experiment The 
source of such responses has not been a matter of primary interest to 
students of instrumental conditioning, but several possibilities have been 
suggested These have included (a) innate or previously learned responses 
to drive, and (b) responses generalized from previous learning because 
of similarities between the two situations. In many experimental studies 
special training has been used to evoke the first few responses. 



Classical and Instrumental 
Conditioning Compared 

The origin ah distinction between instrumental and classical condition- 
ing is made on purely operational grounds. T he two designations ref er 
respectively to training procedures in which the response of the subje ct 
does and does not determi ne whether the TICS appea rs. With this distine- 
t!on~established, the question naturally arises as to whether the two pro- 
cedures also lead to different forms of lear ning. An adequate answer to 
this question requires that classical and~instrumental conditioning be 
separately examined to determine basic differences and similarities. This 
is a research program which is simple in conception, but extraordinarily 
difficult to carry out. Some of the problems are these : 

1. It is impossible, on logical grounds, to perform an instrume ntal 
cond itioning experiment without also arranging the conditions for a 
cl assical conditioning one. T o obtain instrumental conditioning it is essen- 
tial to reinforce a subject for performing some act. Obviously this rein- 
forcement must occur in some situation. Thus the situational cues are 
conditioned stimuli regularly associated with reinforcement, after the 
manner of classical conditioning. The importance of the classical condi- 
tioning accompaniments of instrumental learning has been clearly recog- 
nized by learning theorists and given formal status in such conceptions 
as secondary reinforcement, secondary drive, and incentive motivation. 
But it should be recognized that these processes complicate the problem 
of separating classical and instrumental conditioning in order to compare 
their properties. 

2.. It is also difficult, if not impossible, to perform a complet.Fdv pur e 
classical co ndi tioning experiment, wTtFlhe result th at tlie cla ssically 
conditk merLCR m a y .c o nta in instrumental components . For one~thing. in 
an entirely quantitative way, the appearance of an anticipatory CR often 
has an etrect upon the UCS. In the Pavlovian experiment, the anticipatory 
salivary response no doubt changes the taste of the food when it is used 
as a UCS and dilutes the acid when it is the reinforcement. Similarly, in 
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eyelid conditioning an anticipatory blink partially avoids the air puff 
and diminishes its intensity A rather different complication is depicted 
by Zener’s description (p 54) of the “expectant" behavior of the do™, 
in his salivary conditioning study ana by Pavlov’s account of the imes- 
tigatory, orienting behavior commonly evoked by the CS Such behavior, 
quite clearly, is o£ an instrumental sort and is probably modified mstru’ 
mentally during the course of conditioning This, again, means that the 
CR obtained in the classical conditioning situation may be modified by a 
concomitant instrumental response 
3 JEt is difficult to insure the occurrence of th e same responses under 
the two procedures when experimental attemnt s are made to Vnmnnrp 
-Cla ssical a nd instrumental conditioning A senes of such comparisons has 
appeared m the literature, using an unavoidable noxious stimulus as the 
reinforcement in the classical situation and an avoidable one in the instru- 
mental parallel In some cases, the results have been very similar, whether 
or not the conditioned response prevents the shock (Schlosberg, 1934, 
1936, Hilden, 1937, Miron, 1939) In other cases there has been a distinct 
difference, most often in favor of the instrumental procedure (W S 
Hunter, 1935a, AVhatmore, Morgan, and Kleitman, 1946) The best- 
known study in this latter category is that of Brogden, Lipman, and 
Culler (1938) A guinea pig was placed m a revolving cage and, after a 
conditioned stimulus (buzzer), it was given a shock designed to evoke 
running behavior One group of animals, trained according to the classical 
procedure, were shocked whether they ran or not Another group, sub- 
jected to the method of instrumental avoidance training, were not shocked 
if they ran Learning began similarly in both groups, but it reached a 
much higher level of performance in the second group The results are 
plotted in Figure 4 1 Guinea pigs which were shocked whether or not 
the y ran continued to show anticipatory amtalion at the sound 'of the 
lruzzei\ hut after the first few trials the tendency to run did not increas e, 
those shocked only if they did not run developed the habit of running 
promptly at the sound of the buzzer This study, superficially considered, 
KPpmg tn Rlmw that, avoidance c onditioning is superior to its clsssieal court 
ternart Sheffield (1348), however, was quick to point out that such an 
interpretation is unwarranted, for the onset of shock m the classical situa 
tion might occur while the animal was running, thus punishing the anunal 
for the response to be learned Under these conditions, the subject might 
perform and learn responses other than running in the non avoidable- 
shock situation Sheffield repeated the Brogden, Lipman, and Culler ex 
periment, keeping detailed records of what the animal was doing when 
the shock came on His general results were essentially the same as those 
of the original experiment, but detailed analysis of the records showed 
that the animals m the nonavoidable shock condition sometimes ran and 
sometimes stopped running when the shock came on Moreover, t ie *- 
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FIGURE 4.1. Comparison of performance under classical (no avoidance) and 
instrumental (avoidance) conditions. The solid line represents the frequency 
of conditioned responses (running) of 4 guinea pigs during the two seconds 
that a buzzer sounds, when running prevents the appearance of a shock 
which occurs on those trials in which the animal does not run. The broken 
line represents the corresponding values for another group of 4 animals • 
when the buzzer is invariably followed by shock whether or not the animal 
runs. Data from Brogden, Lipman, and Culler (1938). 

havior incompatible with running tended to be repeated on succeeding 
trials. The animals in the avoidable-shock condition more uniformly 
learned to run in response to shock. Thus the animals under the two con- 
ditions learned different things, and comparison with a measure which 
reflects only one of the possible responses cannot reveal the relative 
effectivenesses of the classical and instrumental training procedures. 
Brogden, Lipman, and Culler’s observation that subjects conditioned by 
the classical procedure continued to show agitated behavior in the pres- 
ence of the CS, whereas those in the instrumental procedure did not, is 
important in the same connection. It sounds very much as if an emo- 
tional response were actually better conditioned under the classical pro- 
cedure than under the instrumental one. 

That classical conditioning may lead to better conditioning of at least 
some responses is supported by the results of two studies of the condi- 
tioned eyelid reflex (Logan, 1951; Kimble, Mann, and Dufort, 1955). In 
both of these experiments classical eyelid conditioning was compared with 
an instrumental procedure in which a blink in the CS-UCS interval 
avoided the UCS (shock or air puff). The results were exactly the same 
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in the two studies. The classical procedure was very much superior to the 
instrumental one. The results of one comparison appear in Figure 

/some similarities between classical and 
X INSTRUMENTAL conditioning 

same term conditioning to describe them. Presumaoiy 
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such later discussion. Second, this presentation Trill provide a documenta- 
tion o£ the point that classical and instrumental conditioning are similar 
in many ways- The plan of the section is to present, wherever necessary 
and possible, two examples of each phenomenon discussed, one irom 
classical conditioning and one from instrumental. 

“ 1. Acquisition. The first two phenomena to be described require no 
specific experimental examples, since they will be familiar to everyone. 
Acquisition is the basic process assumed to operate in learning studies. 
It is reflected in changes in several different response measures, the most 
common of which are the following: (a) probability of occurrence, ex- 


pressed as the percentage of trials on which a given subject produces a 
CR, or the percentage of subjects giving a CR on a given trial; (&) 
latency, the time between the presentation of a signal and the occurrence 
of a CR ; (c l response speed, the reciprocal of some time measure such as 
latency; (<2) rate of responding , the number of CR’s produced in some 
standard period of time; (e ). response magnitud e; some measure which 
reflects the vigor of a response on trials when it occurs, and (f ) resistance 
in extinction . Occasional investigators have given different groups of sub- 
jects different numbers of acquisition trials and then have employed the 
number of unreinforced responses which occurred before the attainment 
of some criterion of extinction in each group as a measure of the strength 
of conditioning. Almost any of these measures will reflect the process of 
acquisition, at least under certain circumstances. 

2 . Extinction. Experimental extinction, as Pavlov called it, or more 
briefly, extinction, refers to the decrease of response strength with non- 
reinforcement. Again no specific experimental documentation seems 
necessary. It is a process with which everyone is familiar, in both classical 
conditioning and experimental conditioning. A longer discussion of the 
topic appears in Chapter 10. 

Spontaneo us Recovery. A response which has been extinguished re- 
covers'iome'oTttS' slreugcirwith rest. Pavlov (1927), for example, de- 
scribes an experiment in which a series of i extinction trials reduced the 
number of drops of saliva secreted to the CS from 10 to- 3. As another 


measure of extinction, the latency of the response increased from 3 to 13 
seconds. Twenty-three minutes later the salivary secretion on the first 
test with the CS was 6 drops ; the latency was 5 seconds. Both measures 
show a spontaneous recovery of response strength during the 23- min ute 


In the instrumental conditioning situation, Ellson’s (1938) investiga- 
tion will serve as a reference experiment. He trained rats to press the bar 
in a Skinner box for a food reward, extinguished this response and after 
various periods of time tested the rats for spontaneous recovery, obtain- 
ing the results shown in Figure 4.3. Apparently the process of spon- 
taneous recovery follows a time course. On the basis of statements of 



CLASSICAL AND INSTRUMENTAL COMPARED 


83 


0 «S MO 

0 recovery interval in minutes 

* .V,,. har Dressing reaction as a function 
•— lE,,so “' WS8) - 

Pavlov, it seems very Italy that the “«“ 

pends upon the original s ren D psoonse has been extinguished and 
trials, and the number of tnnes ‘ | iscussion appears in Chapter 10. 
reconditioned. Again a more extensive . eondf 
4 External MnbtUtm Dur „ alteration of tile conditioning 

tioneTresponse is eai^Tniiueneedby « ^ tho cs> foP example, 

procedure. Any new stimn CR pavI( f v referred to this reduction 

tends to reduce the strength stimulus as an external nlTuWo 

as external inhibition and to .the ™ ^ ^ , be CS a usual one 

Evans (1925) describes an espe o£ a gramophone for a fow 

and the external inhibitor was the P 7 ^ assiKn , bc uninhibited 

seconds ” The results are express ^ Table g This , able also sl.ons 

CR a value of 100 per cent They PP inUib i t0 r with repeated 

that the subject may adapt to 
presentation. 

TABLE 5 

~ 

Strength 0/ eteuol CR Sm 


Strength of CK to et.nni ® f"’ e •• 2nd , 


— — “ „ t external inhibition in 

Winmeh and Hunt (•*»£»* rats rece.ted -he —« 

he Grahani-Gogno rumtaj. Differt 




g4 CONDITIONING AND LEARNING 

o£ a buzzer as an external inhibitor just before raising the door (CS) to 
be<nn Trial £ 8, 12, or 14. Table 6 presents a comparison of the latencies 
on these trials together with the latency on the immediately preceding 
trial without a buzzer. These results show that the effect of an external 
inhibitor decreases with increasing strength of the learned response. This 
also happens in Pavlovian conditioning. 


TABLE 6 

Mean Latent Period in S econds on Buzzer and No-Buzzer Trials 

Buzzer Trial Latency No-Buzzer Trial Latency Difference 

31.40 1 9 - 12 22-18 

17.40 7 8.13 9.27 

6.90 11 4.27 2 .63 

3.89 13 2.30 1.59 


2 

8 

12 

14 


5 Bidnhibition . In the experiment just described, Winnick and Hunt 
also studied the effect of the buzzer during extinction. The four groups 
which had had the buzzer presented on Trials 2, 8, 12, and 14 during 
conditioning received it respectively on Trials 2, 4, 5, and 6 of extinction. 
Presentation of tbe buzzer on Trial 2 produced a signficant decrease in 
latency ; that is, a strengthening of the response. On the other trials, the 
extra stimulus had no discernible effect. 

'When Pavlov observed such a phenomenon in classical conditioning he 
called it disinhibition or inhibition of inhibition. In several studies of 
this phenomenon reported in the 1927 English translation, Pavlov used 
natural reflexes to the sight of food. In one example, the meat powder 
presented at a distance produced 11, 4, and 0 drops of saliva on 3 suc- 
cessive trials. In short, the salivary CR to the sight of food extinguished. 
On the fourth and fifth trials, meat powder was presented again, but, on 
these occasions, tactile stimulation of the skin or an auditory stimulus was 
presented simultaneously. The response strength on these trials increased 
to 2 drops of saliva on one trial and to 3 on another. In short, the extra 
stimulus had the same effect in this experiment as the buzzer did in tbe 
Winnick and Hunt study. Pavlov reported that applications of the un- 
conditioned stimulus were particularly effective in disinhibiting an ex- 
tinguished response, as were other stimuli capable of eliciting the UCR. 
The fact that various methods could he demonstrated for restoring an 
extinguished CR, without pairings of CS and UCS, led Pavlov to the 
position that extinction did not consist of an “irreparable destruction 
of the conditioned reflex,” but that it was inhibited or, somehow, held 
in check. It was for this reason that Pavlov sometimes referred to dis- 
inhibition as inhibition of inhibition. 

6 - jl of Delay . In Pavlov’s long-trace and long-delay experi- 
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ments the interval between the onset of the CS and the administration of 
e was sometimes several minutes In such cases the conditioned 
response, when well established, displayed a characteristic pattern No 
response occurred until the CS had been on for a while, and the amount 
of salivation became greater as the time of presentation of the UCS came 
nearer In one study the CS was a whistle of 3 minutes duration The 
UCS was acid After considerable training the number of drops of sain a 
occurring in each of the 30 second periods between onset of CS and the 
presentation of the UCS were 0, 0, 2, 2, 3, 6 
The instrumental parallel to this pattern is to be found m the per 
formanee of subjects conditioned on a fixed interval (periodic) schedule 
In such a schedule, the subject is rewarded for the first response (for 
example, a bar press or a key peck) after a specified period of time An 
example of the sort of response pattern obtained under such schedules 
appears m Figure 4 4 This pattern has come to be called a periodic' 
reinforcement or fixed interval “scallop ’* The similarity of this to the 
salivary data mentioned above is obvious 

Whether this is more than a superficial similarity depends upon 
whether the two processes respond in the same way to the same experi 
mental operations. In at least one respect, they do The extent of the 
period of inactivity in the fixed interval scallop depends upon the dura 
tion of the interval The same is true m the classically conditioned salivary 
response A more important test of whether the two functions are 
basically the same is potentially provided m the fact that the sahxarj 
response can be dismhibited during the early parts of the period of delaj 
An example of this comes from an experiment in which the CS is a tactile 
stimulus of 3 minute duration A representative response pattern to it for 
successive 30 second intervals is 0, 0, 0, 8, 10 and 11 drops of saliva 
The same CS presented together with a metronome elicited 3 2,1 5 6 
and 5 drops on the next trial The effect of the metronome was to disin 
iuiui the response during the first 1 5 rmnntes nn this trial and to ex 
ternalJy inhibit the Clt for the last 15 minutes Pavlov found that the 
precise effect of an extra stimulus presented this wa> depended upon the 
intensity of the added stimulus Very weak stimuli had no effect at all 
Moderately weak stimuli dismhibited the salivary response in the first 
part of the period of the CS Moderatclj strong stimuli dismhibited 
saliv ition during the imtnl phase of the CS and inhibited it later, is in 
the example above 1 Very strong stimuli had no effect in the initial por- 
tion of the period and produced onlj the later inhibitor} effect Teat* for 
similar effects during responding on a fixed interval schedule have not 
been made Tins lack suggests an important area for future exjverwnenta- 
tlQH 

a Yicnitr (1930a) a »lmitar mult with the tooditicoed GSlt. 
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FIGURE 4.4. The development of a typical fixed interval “scallop.” The in- 
terval employed was three minutes. That is, the first response to occur three 
minutes or more following the previous reinforcemnt was reinforced. Early 
in training (as in A) the actual interval between reinforcements is typically 
more than three minutes because the animal does not respond rapidly enough 
to obtain reward as soon as it is available. With increased rates of responding 
(as in B) rewards occur oftener, and the typical behavior is a fairly steady 
rate of responding, although “scalloping” occasionally occurs. Still later (as 
in C) a pronounced scallop may appear. (After Skinner, 1938; as presented 
by Keller and Schoenfeld, 1950). 
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7 Summah 0 M_Vrhen two stimuli to which the same response lias been 
separately conditioned are presented together, the strength o£ the re 
sponse is often greater than it is to either stimulus alone This is called 
summation The following Pavlovian example Illustrates the process 
One CS was the odor of oil of camphor Presented alone it onimarili 
evoked 60 drops of saliva The other stimulus, a mild shock, ordinarily 
produced about 30 drops Presented together they evoked SO drops Such 
linearity in the summation effect is not the rule Most often the effect of 
the combined stimuli is less than would be predicted from simple addition 
of the individual response strengths 


An experiment of summation, using an instrumental procedure, was 
performed by Bmuger (1952) He trained rats to run a T mare for food, 
requiring them to take a right or a left turn at the choice point on differ 
ent trials m response to different signals For one group the signal was 
visual the subject was required to make a right turn after passing 
through a black maze stem and a left turn after passing through a white 


maze stem For another group the right turn was correct when a tone 


was present, and the left turn was correct when it was not A third 
(summation) group received both the visual and the auditory cues The 
number of trials required for each group to attain a criterion of 90 per 
cent correct choices were Tor the visual group, 148, for the auditory 
group, 234, for the summation group, 55 The marked superiority of 
this last group illustrates the operation of a summation effect 
8 Stimulus Generalization A response conditioned to a certain stirau 
lus tends to appear to similar stimuli to a degree which diminishes with 
increasing differences between the conditioned stimulus and the test 
stimulus. The study of Bass and Hull (1934) will provide an illustrate c 
example using classical conditioning procedures These investigators con 
ditioned the GSR’s of 16 college students using a ubi ^liU&Jc stimulus as 
a CS and a brief shock as the UCS In conditioning the CS was applied 
^to the shoulder or to the calf for different groups of subjects After such 
conditioning all subjects received tests with the CS applied to the 
shoulder, back, thigh and calf In order to combine data for the too 
groups, responses when the CS was applied to the shoulder for one grot/p 
were averaged with those obtained when tlic CS was applied to the calf 
for the other, these being the points of application of the CS for the two 
groups respectively Similarly responses to the CS applied to the back for 
first group were combined with those to the CS applied to the thigh for 
the second group, these being one step awaj from the CS point for the 
two groups This combining procedure makes it possible to prevent all of 
the data for the experiment m a single graph such m that m Figure 4 » 
The horizontal axis represents the four tot stimuli, the vertical axis » a 
scale of response strength in terms of average deflection' of the fcal'anom 
etcr needle The term stimulus gcncraluatwu nfera to the fact that the 
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STIMULATED POINTS 


FIGURE 4.5. Stimulus-generalization gradient which shows the spread of the 
GSR conditioned to tactile stimulation of one point on the body to others. 
See text for fuller description of the experiment and manner of plotting data 
(M. 3. Bass and HuU, 1934). 

response spreads from the CS to adjacent stimuli. The function repre- 
sented in the graph is often called a generalization gradient. 

The procedures described by Guttman and Kalish (1956) 'will provide 
an illustration of generalization in an instrumental situation. In this 
method pigeons are trained to peck at a round key on the wall to receive 
food. The key is illuminated with light of a certain color. After training 
the birds are tested with other colors on the key, the rate of pec kin g 
providing the measure of response strength. A typical set of results 
appears in Figure 4.6. Obviously the results show all of the general 
features of similar data obtained by classical conditioning procedures. 

9. Respon se Generalization. The response acquired in any conditioning 
experiment has a modal value determined by the exact conditions of the 
experiment and the capabilities of the organism. It has been pointed out, 
however, that an organism which has learned to respond in a certain way 
to stimulus A, has thereby learned also to respond in a similar way 
to an almost equivalent stimulus A'. The counterpart on the response side 
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WAVE LENGTH (M/i) 

FIGURE 4 6 Generalization of response strength along a dimension of hue 
in the pigeon The birds were trained to peck at a key illuminated with a 
light of 550 mu and tested with other lights in extinction. (Cullman and 
Kahsb, unpublished) 

ts as follows if an organism has learned to react with response B to a 
stimulus A, jt has also learned thereby to react with a response B', which 
is unlike B, but in some respects equivalent to it One illustration comes 
from Bekhterev (1932, p 216) who found that a dog lifted another foot 
to the conditioned stimulus when the one winch normally responded was 
fastened down Another instance is furnished by LashU) 's (19Ji) 
monkey, which quickly solved a problem of manipulation with the other 
hand after the hand which had always been employed previously was 
paralyzed due to a brain operation Further observations of the kind 
suggested by Bekhterev have been reported by Kellogg (1939) Following 
buzzer shock conditioning m which the shock was applied only to the 
right hind foot of a dog, it was found that each of the four legs occasion 
ally responded to the buzzer Responses m the more remote legs were less 
frequent than those in the shocked leg, the relative frequencies of condi 
turned responses being m the order right hind leg (the one ihockcd), 
left htud leg, right foreleg, left foreleg This represents a crude gradient 
of response strength, corresponding somewhat to the sensory gradient* 
previously described 
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A mote detailed picture o£ the response-generalization process 
closer inspection o£ the exact characteristics o£ the response. Sue P 
tion always reveals quantitative variation m response stien„th. 

Tt shlws two examples of this £or the Skinner box situation. The data 
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INTENSITY OF PRESSURES IN GRAMS 


FIGURE 4.7. Tlie distribution of bar-pressures under conditions where 21 gm. 
of pressure (upper figure) or 38 gm. of pressure (lower figure) were required 
to obtain food. Note the variability in pressures (response generalization) 
and also the fact that the modal pressure is such as to certainly secure rein- 
forcement (response diSerentiation). Data from Ruth Hays and C. B. Wood- 
bury, presented in this form by Hull (1943). 

are simply the pressures with which individual rats pressed the bar on 
a series of 100 responses. In one case a pressure of 21 grams was required 
to secure food reinforcement ; in the other it was 38 grams. This differ- 
ence accounts for the differences in the functions. 

Figure 4.8 shows a set of similar data for the classical eyelid- 
conditioning situation presented by Boneau (1958). The data are distri- 
butions of CR latencies in blocks of 30 trials. Obviously the classically 
conditioned reaction shows the same sort of distribution on a quantitative 
dimension as did the instrumental response. 
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RESPONSE LATENCY 

FIGURE 4 8 The distribution of response latencies in eyelid conditioning 
The interval between the CS (light) and UCS (airpuff) was 1 0 second. The 
several panels in the graph, from upper to lower, are for fifths of the Condi 
boning session Note that there is considerable variability in the latencies ol 
the various responses (response generalization) Also note the tendency for 
the median, indicated by the arrow, to mine to the left (response differentia 
hoa) so that t lie blink is timed to just anticipate the onset of the VQ> 
(Boneau, 1958). 

10 liemonsc Differentiation I n each of the previous example* the 
mam interest was m the spread ot responses or m tht.tr generalization 
along these quantitative dimensions Eich example however shows 
another characteristic of the responses obtained in conditioning expen 
incuts, namely, that they come to form a distribution whose cmtral 
tendency is dictated bj the requirements of the situation Note that the 
distributions of pressures in Figure 1 7 are sue U as to insure that the 
subject receives reinforcement on most trials Similarly, the median 
latency of the blink changes with practice so that it just anticipates the 
onset of the nir puff Pavlov reports a similar phenomenon for long trace 
salivary conditioning Tins phenomenon is sometimes referred to as 
response differentiation 

The most extensive discussion-, of response difftnntiatipn have been 
those of Skinner and Ins associates (b Winner, 1918 , h erstvr anti bkmm r, 
1957) w ho report studies in winch restricted > alius of in trinity, dura 
tion, and rotes were reinforced These investigators find that rats am f 
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pigeons are capable of learning to modify these quantitative features of 
their behavior in order to maximize the probability of receiving rein- 
forcement. 

11 . Generalization of Extinction . In the Bass and Hull study previ- 
ously described to illustrate stimulus generalization, a second procedure 
was employed to show the operation of a different phenomenon. The con- 
ditioned response was first established at all of the four reinforcement 
points (shoulder, back, thigh, and calf) so that the magnitude of the 
response was approximately the same at all points. Then one of the 
extreme points (shoulder for one group, thigh for another) was sub- 
jected to extinction. Following this all points were retested. The results 
appear in Figure 4.9, in the form of a generalization gradient. Clearly 



FIGURE 4.9. The generalization of extinction. Responses were fir 
Honed to all stimulated points. Then extinction was carried ou atll 
and tests were made at the others (M. J. Bass and Hull, 1934) 


the effect of extinction at point 0 spread or generalized to the adjacent 
points a manner exactly analogous to the generalization of excitation 
Pavlov referred to the process, in cortical terms as tho y/i 
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Youtz (1939) performed a study with rats which showed that extinc- 
tion also generalizes m instrumental conditioning Nineteen rats were 
trained to press two bars in the Skinner box, one a vertical bar and one 
a horizontal bar After the rats had learned to operate both bars, he 
extinguished the two responses one after the other The average number 
of trials required to produce extinction was 43 for the response extin- 
guished first and 16 for the response extinguished second This reduction 
of 63 per cent shows that the effect of the first extinction generalized and 
aided in the extinction of the second 


12 Discrimination If a subject is reinforced for responding to one 
stimulus and not to another, a discrimination between the two gradually 
develops The end result of this process is that the subject comes to 
respond consistently to the reinforced stimulus and not at all to the 
nonreinforced one Pavlov reports numerous examples of this for sain ary 
conditioning There are also many illustrations available in the Iitcra 
ture on instrumental conditioning Examples appear in the section on 
induction below 


13 Conditioned Inhibition A special form of discrimination studied 
by Pavlov was what Jie referred to as conditioned inhibition In such 


studies the positive CS was a single stimulus, such as a tone The negn- 
tive CS was the same stimulus in combination with another stimulus. 


perhaps a tactile one After training the dog came to respond to the 
single stimulus, but not to tbe combination Because the added stimulus, 
through conditioning, came to interfere with salnation, Pailov called it 
a conditioned inhibitor 


Woodbury (1943) has performed a similar studj m an instrumental 
situation Dogs were placed m a wooden stock and trained to lift a bar 
with the nose to obtain food After this initial training, thej were pre- 
vented from performing this response until after the presentation of a 
signal, which on different trials might be a high pitched buzzer, a low 
pitched buzzer, or a combination of tbe two stimuli In the portion of 
the Study most nearly resembling the PoUo\jan prototype, the dog 
learned to respond to either tone m isolation, but not to tbe combina 
tion The course of this learning is presented graphically in Figure 4 30 
The scry slow learning probably depends upon two factors (1) all of 
the stimuli were in the same modality, creating a problem in sensory 
discrimination, (2) since three stimuli were used the problem was more 
difficult than the Paxlouan demonstration lnsolwng only two 
14 Induction The growth of a discrimination is accompanied b> the 
simultaneous deffclopment of a set of complicating phenomena referred 
to by Paslov as positive and negative induction Poutixe induction refer* 
to the augmentation of an excitatory process b> a preceding inhibitory 
one; mgatue induction refers to the intensification of an inhibitor} 
effect b} a preceding oxutJtorj one In this usage, the excitatory an 
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inhibitory processes are those associated, "with the reinforced and non- 
reinforced stimuli in a discrimination experiment. 

The example summarized in Table 7 provides an illustration of posi- 
tive induction. The positive CS was a tactile stimulation of the front 
paw ; the negative CS was tactile stimulation of the hind paw. Both were 
of 30-seconds duration. The inductive effect occurs on the fourth trial in 
the table. It consists of an increase of approximately 50 per cent in the 
strength of the salivary response and a decrease in latency. This example 
also illustrates certain other typical features of positive induction: (a) 
To obtain it, it is necessary to test with the positive CS immediately after 
the application of the negative stimulus, (b) Following the response 
augmented by positive induction, the next trials tend to elicit reactions 
of lesser magnitude and of longer latency. This was taken by Pavlov as 
evidence for the existence of a slower acting inhibitory aftei'-effect. Posi- 
tive induction seems to occur with greatest intensity in newly established 
discriminations and with discriminations involving somewhat similar 
stimuli. It may completely disappear after protracted discrimination 
training and may fail to develop when the positive and negative stimuli 
are very different. 

Demonstrations of induction in instrumental conditioning experiments 
are difficult to find. There is one study, however, (Hanson, 1959) which 
demonstrates what may be an instrumental parallel to the process of 
positive induction. This investigation was concerned with the form of the 
stimulus generalization gradient following discrimination training. The 
procedure consisted of training pigeons to make a hue discrimination, 
pecking the key m a Skinner apparatus when it was illuminated with 
one color and not when it was illuminated with another. After this dis- 
crimination was established, the pigeons were tested on a ran«e of colors 
Among other things, Hanson found that response strength, measured in 
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tiese tests, was considerably raised to stimuli other than the negative CS 
or those like it, by comparison with control groups tested without dis 
crimination training See Figure 4 11 



FIGUKE 4 1L The effect of discrimination training upon stimulus gencruliza 
lion in the pigeon The birds were trained to peck at a key illuminated with 
a light of 550 mu, and not to respond when the light ( for different groups) 
had a hue corresponding to 555, 560, 570 , or 590 mu The control group mi 
trained to peck at the key illuminated with a light of 550 mu, but it received 
no discrimination training Generalization teas were conducted in extinction 
Observe the greatly increased strength of response to tumuli to the left of 
the CS on the hue dimension This may he an instrumental parallel to the 
phenomenon of induction described by Pavlov for classical condiuontng. 
Data from Hanson (1959) 


TABLE 7 


Demonstiution of Positive Indlction 


Time CS 

Mtuofton ih 

SO seconds 


4 20 Iront jaw 

S 

3 

4 30 i ront paw 



4 45 Illnd paw 



4 45*i iron* paw 



4 53 Front paw 



5 10 Front paw 

0 •/ 
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Pavlov also described a process of negative induction and seemed to 
regard it as exactly opposed to positive induction. This conception, how- 
ever, has been criticized (Konorski, 1918) on the grounds that the two 
kin ds of demonstration were very different. The original evidence for 
negative induction involved an attempt to extinguish a conditioned in- 
Mbition. In this experiment (Pavlov, 1927, pp. 196-198), the positive 
CS was a tone and the negative CS was the same tone in combination 
with a tactile cutaneous stimulus. Previous to the portion of the study 
to be described here, a perfect discrimination between the two had been 
established: the dog always salivated to the tone alone, never to the 
combination of tone and tactile stimulation. The attempt to extinguish 
the conditioned inhibition consisted simply in reinforcing the combina- 
tion with acid in the same way as the single tone was reinforced. In a 
series of 23 trials, carried out in three sessions, with tone trials and 
combination trials alternated, the conditioned inhibition failed to extin- 
guish. That is, the dog continued to salivate to the single tone but did 
not begin to salivate to the combination. It was as if the reinforced trials 
with the tone alone deepened the inhibition already associated with the 
combination of stimuli. When the experimental procedure was changed 
and the previously negative combination of stimuli was reinforced on 
successive trials, the discrimination broke down on the second trial and 
the dog salivated to the combination of tones. No instrumental analogue 
to negative induction appears to have been demonstrated. 

15. Higher-Order Conditioning . Under certain conditions, Pavlov 
found it possible to use the conditioned stimulus from one phase of an 
experiment as the unconditioned stimulus for a further conditioning of 
the salivary response. 2 In one demonstration the original CS was an 
auditory stimulus. When the CR to this stimulus had been very firmly 
established, a black square was presented briefly before it. On the tenth 
pairing of the black square and the auditory stimulus a salivary re- 
sponse, about half as strong as the response to the auditory stimulus, 
occurred to the square. This is an example of higher-order conditioning, 
specifically a conditioned reflex of the second order. Pavlov found third- 
order conditioning possible, but only with defense reflexes such as those 
in response to shock. Fourth order reflexes could not be established in 
dogs. 

Typically the higher-order CR’s were small in amplitude, long in 
latency, and had only a short life span. This last fact follows ’from the 
consideration that the higher-order conditioning procedure is the same 
as that for establishing conditioned inhibition. In short, a singly-presented 
stimulus is reinforced ; the same stimulus in combination with the higher- 
order CS never is. Because of this, trials with the paired stimuli lead, 

‘See also Brogden (1939), J. V. Murphy and Miller (1957), Bazran (1955a). 
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Zr^ CStabl ‘ SIiment of tLe secondary reflex, to its gradual 

The last that the secondary conditioned reflex is a somewhat evanes- 
cent phenomenon does nothing to diminish its theoretical importance 
which IS that the reinforcement (CCS) m the higher order conditioning 
procedure obtains its reinforcing power as a result o£ learning The 
secondary conditioned reflex, in modern terminology, is established on 
the basis of a secondary reinforcement (or secondary reward, or acquired 
reward) The process of secondary reinforcement has been of much 
greater interest m the field of instrumental learniug than classical 
conditioning 

The basic procedure for providing a neutral stimulus with secondary 
reinforcing power is to pair such a stimulus -with another stimulus which 
already is a reinforcer Stimuli regularly associated with food or water, 
for example, become secondary reinforcers The evidence that these, 
originally neutral, stimuli now have reinforcing properties most often 
consists of a demonstration that the secondary remforcer will function 
as a reward in the learning of a new response Wolfe’s (193G) study of 
token incentives m the behavior of chimpanzees illustrates this type of 
demonstration The tokens were small disks similar to poker chips to 
which the animals were originally quite mdifTcrcut Preliminary training 
consisted of teaching the chimpanzees to insert tokens m the slot of 
a vending apparatus which automatically released a grape to the animal 
After having learned the reward value of tokens, the chimpanzees were 
trained to perform simple instrumental responses to secure the tokens 
One task required the lifting of a lever, another involved pulling in a 
small sliding tray by means of a cord The animals would continue to 
•work at these tasks for a number of tokens which could not be exchanged 
for food until later In an extension of these experiments, Cowles (1937) 
found that token rewards were effective as rewards for the acquisition of 
simple position habits and visual discriminations. A position habit re- 
quinng as many as 20 trials could be complete 1} learned in one session 
with tokens as the sole reinforcing agent and, therefore, prior to any 


reception of food reward 

Secondary rewards such as approval, money, prestige, and so forth are 
of unquestioned importance m directing most learning by adult human 
individuals In the typical learning experiment with college students the 
only reward provided is the knowledge of which respond arc correct 
and which incorrect The mechanism of secondary reward training u 
difficult to analyze m the case of human learning because of the long 
and complicated history of learning through which such reinforce on* 
develops The essential features of the procedure hive, however, 
successfully demonstrated with animals such as rats, cat*, and degu ** 
well as with chimpanzees. The results m general indicate that a secorJary 
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reward is somewhat less effective in the establishment of new learned 
responses than the primary reward with which it has been associated. 
Moreover, the secondary reward loses its reinforcing value (extinguishes) 
if it is not itself repeatedly followed by the primary reward. In the 
chimpanzee experiments it was found that a token which was immediately 
exchangeable for food was more effective as an incentive than tokens 
which were exchangeable only at the end of a series of trials. 


DIFFERENCES BETWEEN CLASSICAL AND 
INSTRUMENTAL CONDITIONING 

The materials in the foregoing section amply testify to the fact that 
the similarities between classical and instrumental conditioning are suffi- 
ciently impressive to make the assumption that the two processes are 
basically the same at least tenable. There is, however, an alternative pos- 
sibility which should not be overlooked. As we have seen, it is impossible 
to perform an experiment involving instrumental conditioning without 
at the same time arranging the circumstances under which classical con- 
ditioning occurs. Thus, the similarities just catalogued may occur because 
of a classically conditioned component common to all learning. Generali- ' 
zation, according to such an argument, might actually be a phenomenon 
of classical conditioning exclusively. But, because of the inevitable ele- 
ment of classical conditioning in all learning, generalization seems to 
occur in instrumental learning, too . 3 Although the details of such an 
explanation have not been worked out, the possibility that one could be 
developed means that the existence of parallel phenomena does not prove 
that classical and instrumental conditioning are different varieties of 
the same basic process. To be completely clear on this point, the bulk of 
the evidence suggests that classical and instrumental conditioning are 
different forms of learning; but a different view is more than barely 
possible. 

Various investigators taking a less extreme position have held that 
classical and instrumental conditioning are both forms of learning but 
that they differ in one or more important detail. The most important 
points of difference which have been proposed are the following : 4 


3 Among the individuals whose writings suggest a development of this sort are 
Spence (for example, 19o6) whose use of the incentive concept represents the intro- 
duction or a classical conditioning component in instrumental learning Tolman C194Q1 
whose separation of expectancies (classically conditioned) and motor patterns (instru- 
mentally conditioned) does not at all deny the concept of these processes’ goin? on 
together, Solomon and Brush (1956), Mowrer (1947), Beese fl95Vi 
and Collier and Siskel (1959). V h (1953) ’ Welch ( 195o )> 

‘Discussions of the differences between classical and t . 

"'‘“'I the too in terma of the.a or xel.ted eategoxtoVwT'b! 
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I Some investigators have suggested that classical conditioning a 
.sti mulus stimulus le arning and that instrumental learning is stimulus- 
response nam i n g Schlosberg has maintained this view since 1937 The 
following description of his position is a slight paraphrase of a recent 
personal communication in which he puts his position very plainly 


I prefer to think of classical conditioning os peculiarly appropriate to setting up 
preparatory responses This leads to a tbeoiy of stimulus substitution in the sense 
that Pavlov meant it, that is, signalizing, which is fundamentally S S learning 
Thus, I am arguing for S S learning, but sophisticated to the orient that, in 
avoidance conditioning for example, the conditional stimulus stands for “shod 
about to come,” so that the animal makes all sorts of responses which are more 
or less appropriate m Nature for shock about-to come Instrumental condition 
mg fixates the response which is effective m avoiding the shock I would have no 
objection to speaking of some kind of cortical activity as set up by classical 
conditioning, but I certainly would not want to call it a response 


The last sentence in this passage is basic Other two process theorists, 
such as It L Solomon, have held a similar position, with the exception 
that the classically conditioned process was assumed to be a fear response 
having drive properties capable of evoking what correspond to Schlos 
berg’s “all sorts of responses which are more or less capable of shock- 
about to come m Nature ’’ Otherwise these other theories and Schlos- 
berg ’s tend to be much the same 

2 As a part of the two process theory it has vciy often been proposed 

that different basic laws of learning apply to classical end to instrument al 
conditioning Specifically the usual position is that the primary Jaw of 
classical conditioning is the law of contiguity , whereas instrumental 
conditioning involves, in add ition, a law of effect^ Thus, classical condi 
turning is thought by such theorists to occur as a result of the mere pair 
mg of the conditioned stimulus and either the unconditioned stimulus or 
the unconditioned response A detailed discussion of the validity of his 
position will have to be postponed (see pp 267 to 277) for three retsorii 
(a) An adequate treatment of the question cannot very well be prewnted 
without a more extensive discussion of the law of effect than is feral hie 
here (b) The data relevant to the two process theory are also bctftr 
considered in connection with the general problem of reinforcement In 
anticipation of a lengthier discussion, it rany be said that the hulk of 
the evidence supports the two factor theory (c) It possible to 

establish the fact that the two forms of learning differ in one bisic regret 
on other grounds to be considered later 

3 It has sometimes becu suggested that an important difference be- 
tween classical and instrumental conditioning 

similarity of the conditioned and Mn<ninfitmncdrc»fmn^« OhLsrd a: J 
5larq uIsTT55o ) More "specifically this proposition is that in 
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conditioning, the conditioned and unconditioned responses are always 
the same, whereas in instrumental conditioning they are always different. 
Although this observation is approximately correct, it is not exactly so 
in either of its two particulars. For (a) in classical conditioning the CR 
and UCR are only grossly similar. They involve the same effectors, but 
as we have previously seen, the responses are found to. differ when sub- 
jected to close inspection. And (6) although the conditioned and uncon- 
ditioned responses can be very different in instrumental conditioning 
(for example, bar pressing as opposed to eating in the Skinner situation) 
they can also be very similar. The typical shock-a voiding condi tioned 
flexion responses are good examples of this. 

JTOne aspect of certain of the two-process theories has been the pro- 
posal that the organism’s responses are of two basic sorts, w hich differ 
in the learning processes by which they can be modified. The clearest 
statement of this position appears in Skinner’s (1938) distinction be- 
. tweenelicited a nd emitted behavior and in his hypothesis that these two 
forms of behavior are subject, respectively, to , respondent (classical) and 
operant (instrumental) conditioning. In addition to this, two-factor 
positions have often held that the classically conditionable responses ar e 
“ involuntary” responses of the autonomic nervous system and that 
instru mentally conditionable responses arp “voluntary” responses of 
tfie~eentral nervous system. T he evidence, although generally favoi*able 
to such a dichotomization, raises certain difficulties, specifically : (a) there 
are some nonautonomic responses which can be condi tioned classically 
and (6) th e voluntary-involuntary dichotomy is diffi cult to support on 
o bjective grounds. ~ 

One fact which supports the division of responses into dichotomous 
categories is the following. Although autonomically mediated reactions 
such as the GSR and vasoconstriction are readily conditioned classically, 
they seem to be impossible to condition by instrumental methods. The 
Pavlovians simply state (without documentation available in English) 
that glandular responses cannot be conditioned instrumentally . Mowrer 
(1938) was unsuccessful in an attempt at instrumental conditioning of 
the GSR, and Skinner (1938) reports that he and Delabarre could°not 
condition vasoconstriction by making reinforcement contingent upon 
the response. Thus, for_autonomically mediate d behavio r, the evidence 
pomte unequivocally' to~ the conclusion that" such responses can be modi- 

fiedoy~cTassical, but not instrumental, training methods. ' — 

Such evidence does not, however, establish the fact that classical condi- 
tioning succeeds only with autonomic responses. There are, in fact 
several nonautonomic reactions which can be conditioned classically. 
These include the knee jerk, various flexion responses, and the eye-blink. 
These responses are of special interest because they seem at first to 
violate the rule that responses which are classically conditionable cannot 
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be modified instrumental^. It is obvious that, outside the laboraton- 
these reactions often do serve an instrumental purpose, as in lickin'* a 
football, avoiding a painful object, or winking at a member of the oppo- 
site sex. This seems to indicate that they can also be modified by instru- 
mental conditioning. 

Careful consideration of the evidence, however, suggests an alternative 
explanation: that the classically and instrumeDtally modified forms of 
jjiese res ponses are different reactions D ata in support of this contention 
are most complete for the conditioned eyelid reflex. Detailed investiga- 
tions of the form of the response have yielded the following significant 
facts: (a) the instrumental (“voluntary*') response which might be 
employed in winking at another person or in deliberate response to an 
approaching object is quite different in form and latency from the condi- 
tioned blink based on a puff of air blown into the eye (Spence and Ross, 
1959) and (6) the innervation of the eyelid is quite complex enough to 
mediate more than one variety of eyelid response 
Both Marquis and Porter (1939) and H E King and Landis (1913) 
have compared the progress of eyelid conditioning with procedures in 
which the UCR was elicited by an air puff for one group of subjects and 
an instructed voluntary response to a click for another The CS m each 
of these experiments was a light Although conditioning occurred using 
the voluntary response as a UCR, there were significant differences be- 
tween the response conditioned in this way and that obtained using a 
reflex blink. Conditioned voluntary responses were found to bo brisk, 
complete lid closures; conditioned involuntary responses wore smaller 
and developed more slowly with practice 5 This adds to the evidence that 
voluntary and involuntary blinks are probably different reactions. 

A more critical test of this conception would be provided if a way 
could be found to limit the eyelid response by some procedure to just 
the voluntary or just the involuntary variety Under such conditions it 
should be possible to show that only one kind of learning is possible 
Although precisely this experiment has not been done, those of Logan 
(2951) and of Kimble, Mann, aDd Dufort (1935) (p 81), in which 
instructions were employed to eliminate most voluntary blinks, come 
close to meeting the initial requirement The subjects in these com- 
ments did show some acquisition (Figure 4 2) under the instrumental 
conditioning procedure, but the amount of conditioning was small and 
may have resulted from the classical conditioning trials on winch a CU 
failed to occur. 


• Similar characterise* Jure hcca ttuni for cm. Mm* Jw 

ttited with a voluntary rcaet.oa « the UCJt ('«'*»’«* , Cutfcne. 

1933a) In general auch “caaJiUancU repon*-. lead t*U \ * U * 
tary rehouse, and there U no gradual fa»crraa« io amplitude with txaiaie* 
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conditioning, the conditioned and unconditioned responses are always 
the same, whereas in instrumental conditioning they are always different. 
Although this observation is approximately correct, it is not exactly so 

“ ltS t T° particulars - For <«> in classical conditioning the CR 

and UCK are only grossly similar. They involve the same effectors, but 

tT, pre . Ylously . se “- tha are found to differ when sub- 

jected to close inspection. And (6) although the conditioned and uncon- 
ditioned responses can be very different in instrumental conditioning 
(for example, bar pressing as opposed to eating in the Skinner situation! 

they ca n also be very similar. The typical shock-avoiding condWoned 
flexion responses, are good examples of this. e_conompned 

potltoTlg^ baa ^ the pro- 

in the learninv'l^S7lfrS^ differ 

statement of this position appears in SkiTert U^Wi 

(instrumental) conditioning ^ IdlS^T^ (classical) and 
positions have often held that tho nl ° • ddltlon to this, two-factor 
-involuntary- ^ R eally 

instru mentally conditMirTT ^l^ 0 ™ 10 that 

t he central nervous svgt am — ^^ol^uitaryl’ responses of 

to^iehmdlehotomLzation, raises ceEEE" 11 g6nerally favorable 
are 5ome^onautonomic response* wtmi^? 68 ’ specifically: (a) there 
and W thg^ yoluntary -lu vnhi u t n rv classically 

objective gro^dT~ ~^ — ^gS^Jl^ ^no~supp Mt on 

categories is the^folSn g" lltho^h^autf into dichotomous 

such as the GSR and vasoconstriction ^ °™ lcally mediated reactions 
they seem to be impossible to condition 3 read % conditioned classicaUy, 
avlovians simply state (without do™ ^ lnstrument al methods. The 
^^feASSBonses ^not be ia English) 

Uh38) was unsuccessfuThTl^r^r ^ Ded - astrumentaIIy. Mowrer 

the GSR, and Skinner (1938) reports ^ strument al conditioning of 
condition vasoconstriction by mLnt J r “ d Delaba ™ could not 

Tbaa . &r antonomicaJIy med ST of com “S«“ upon 
gj^^cgmyocanyl^ErSdl ausbin ” f , d beha7 - i °r, the evidence 
-jptijgggTbu t not insto^ ^—gtSShjaMgs can be modi- 
Such evidence doeTnoTw! tahjr ainihg methoiTT ~~ 

—-cttsarsSiSaS: 
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be modified mstrumentally It js obvious that outside the laboratory 
these reactions often do serve an instrumental purpose, as in kicking a 
football, atoiding a painful object, or winking at a member at the oppo 
site sex This seems to indicate that they can also be modified by instru 
mental conditioning' 


Careful consideration of tile evidence, however, suggests an alternative 
explanation that the classically and mstrumentally modified forms of 
these resp onses are different reactions Data in support of this contention 
are most complete for the conditioned eyelid reflex Detailed xuvestiga 
tions of the form of the response have yielded the following significant 
facts (a) the instrumental (“voluntary”) response which might be 
employed in winking at another person or in deliberate response to an 
approaching object is quite different in form and latency from the condi 
tioned blink based on a puff of air blown into the eye (Spence and Ross 
1959) and (b) the innervation of the eyelid is quite complex enough to 
mediate more than one variety of eyelid response 
Both Marquis and Porter (1939) and H E King and Landis (1943) 
have compared the progress of eyelid conditioning with procedures in 
which the UCR was elicited by an air puff for one group of subjects and 
an instructed voluntary response to a click for another The CS in each 
of these experiments was a light Although conditioning occurred using 
the voluntary response as a UCR there were significant differences be 
tween the response conditioned in this way and that obtained using a 
reflex blink Conditioned voluntary responses were found to be brisk, 
complete lid closures conditioned involuntary responses were smaller 
and dei eloped more slowly with practice 5 This adds to the evidence that 
•voluntary and involuntary blinks are probably different reactions 
A more critical test of this conception would be provided if a way 
could be found to limit the eyelid response by some procedure to just 
the voluntary or just the involuntary variety Under such conditions it 
should be possible to show that only one kind of learning is possible 
Although precisely this experiment has not been done those of Logan 
(1951) and of Kimble Mann, and Dufort (1955) (p 81) in which 
instructions were employed to eliminate most voluntary blinks come 
close to meeting the initial requirement The subjects in these expen 
ments did show some acquisition (Figure 4 2) under the instrumental 
conditioning procedure but the amount of conditioning was small and 
may have resulted from the classical conditioning trials on which a CR 
failed to occur 


‘Similar characteristics have been found for conditioned finger 
listed with a voluntary reaction as lie UCR (Yacorzmshi and Outlrie . ’ 

1939a) In general such "conditioned responses” tend to be identical with the toJ 8 
iary response, and there 19 no gradual increase m amplitude WJ(h training 
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Voluntary Control over Involuntary Beliavior 

One line of evidence which offers a potential threat to the hypothesis 
that involuntary responses can be conditioned only classically and volun- 
tary responses only instrumentally conies from the well-known, and 
experimentally demonstrated, fact that involuntary responses can some- 
times be brought under voluntary control. This again implies that these 
responses can he modified instrumentally. The range of involuntary re- 
sponses for which there is sound evidence of voluntary control is quite 
wide, including an “invisibly small thumb twitch” (Hefferline, 1959), 
ear movements” (Bair, 1901), the GSR (Noble, 1950), pulse retardation 
(Kotliarevsky, 1936), vasoconstriction (Menzies, 1937; Roessler and 
Brogden, 1943), and perhaps pupillary responses (Cason, 1922a; 
Hudgins, 1933). As Skinner points out, however, it is important to 
determine whether these cases are ones in which the response involves 
a mediating step. Many responses, as we have already seen, are pro- 
ducible in two or more physiological ways, one of which is involuntary, 
the other voluntary. Descriptions of the Yoga procedures which provide 
the most impressive examples of voluntary control over involuntary 
responses suggest that the method makes use of this fact, and that the 
involuntary responses are controlled indirectly through skeletal mecha- 
nisms. In the experimental examples mentioned above, the possible nature 
of the mediating step is usually fairly obvious. Voluntary control of 
vasomotor responses, the GSR, and pulse retardation might require only 
that the subject become a little tense to acquire voluntary control over 
the involuntary process. According to Skinner (1938) controlling breath- 
ing can also mediate voluntary control over these responses. Correspond- 
ing intermediaries may be found for the pupillary response, since the 
pupil is activated, not only by the amount of light falling on the retina 
but by emotional changes and accomodation and convergence as well. 
These latter changes being under a measure of voluntary control may 
mediate the control of the involuntary response. It thus appears that 
there is no evidence which clearly violates the rule that the organism is 
capable of two forms of learning and that they are differentiated in 
terms of the responses to which they apply. 6 


3 A related point of importance which this discussion has nnf 
the extensive literature on “learning without awareness”^ X T'l 

Phillips, 1954; Postman, Adams, fnd PMmSHS , P f tman and 

19o6a, 1956b; Postman, Adams, and Bohm, 1956) and “verbal condithf ^ 
example, Greenspoon, 1955; Taffel, 1955- Cohen Kalish on, + d , t lg ( fc)I 

Verplanck, 1955a, 1955b, 1956). B^th of ’thest moldures Ivow’ ? 4 ° 6he ?* 1954 = 
than classical, conditioning methodology. Particularly in “ strum6ntal > rathe; 

periments, much is often made of theYubiect’s laeW .conditioning ex 

is learning or of the response-reinforcement^ eontSSLl S ” ° £ What * 
above would survive a demonstration of the validitv nf tv, f ^ p0sltlon take5 

■“ 4 « - — -»*. «-* - bis ftrsRS x- 
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Effects of Partial Reinforcement upon die Two Forms 
of Conditioning 


Where comparisons are possible, most of the usual conditioning van 
ables seem to influence classical and instrumental conditioning m the 
same ways The factors for which evidence to this effect is available in 
dude the temporal spacing of conditioning trials, the amount of a posi 
tne remforeer such as food, the intensity of a negative remforcer such 
as electric shock or corneal air puff, and the intensity of the CS The 
most important exception to this rule is to be found m the case of partial 
reinforcement, in which the remforcer occurs on only some fraction of 
the trials Other trials are unreinforced presentations of the CS in 
classical conditioning, or unreinforced responses in instrumental condi 
tiomug In the instrumental situation, very large ratios of unreinforced to 
reinforced trials (for example, 192 reinforced trials to 1 reinforced trial, 
Skinner, 1938) are possible Such procedures may produce a slight 
diminution in rate of learning, but they create a very great resistance to 
extinction In classical conditioning, by contrast, partial reinforcement 
seriously interferes with learning (Razran, 2955a, 1956) Pavlov (2927) 
found that giving food less frequently than every second or third trial 
made conditioning impossible Similarly, m eyelid conditioning, a 50 
per cent reinforcement schedule produces a significant response decre 
ment (W F Reynolds, 1958) Moreover, it has been shown that a shift 
from continuous to partial reinforcement m eyelid conditioning pro 
duces an immediate response decrement (Ross, 1959) This is in contra 
distinction to instrumental conditioning, where a very common procedure 
is to develop the response under continuous reinforcement and then to 
switch to a partial schedule, under which response strength is typically 


maintained intact 

A closely related difference m the effect of a particular variable upon 
classical and instrumental conditioning involves the influence of regular, 
as opposed to irregular, alternation of reinforced and nonremforced 
trials In instrumental conditioning the organism usually shows that it 
detects the pattern by responding appropriately, tending to respond on 
the reinforced and not on the nonremforced trials Nothing of this sort 
occurs in classical conditioning Regular patterns of reinforced and non 
reinforced trials are a particularly destructive form of partial reinforce 
ment schedule, in spite of the fact that human subjects are perfectly 
aware of the properties of the sequence (Grant, Riopelle, and Hake, 
1950) The importance of this difference m respouse to partial remfoice 


t'ary responses are' usually also ones oe winch tie performer is a<rare Lwdeace w 
accumulating, however, (for example, Levin, 1 9o9) that such learning actually docs 
not occur Close questioning indicates that awareness of the condition of rciororccmitn 
was necessary lor learning to occur 
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ment and to patterns of reinforcement is not known. One good possibility 
appears to be that acquisition and, especially, extinction may be more 
rapid in classical than in instrumental conditioning. If so, this would 
mean that extinction occurring in a small number of nonreinforced trials 
would lower tbe level of responding on the reinforced trials. 


The Effect of Voluntary Factors on Conditioned Responses 

Tbe suggestion that tbe classical conditioning process applies only to 
involuntary responses does not mean tbat voluntary processes are of 
no importance in tbe control of sueb learning. J ust as tbe unconditioned 
reflex can be influenced by ongoing voluntary acts, tbe conditioned reflex 
is similarly affected. Indeed sueb variables are probably mueb more im- 
portant in tbe control of conditioned behavior than in tbe control of 
unconditioned responses. 

Razran (1955b) bas routinely found tbat conditioning is better in 
subjects wbo bave no knowledge of conditioning than it is in subjects 
wbo understand tbe process. Several studies of the conditioned eye-blink 
bave shown that the set of tbe subject may bave a marked effect upon 
the level of conditioning attainable. Tbe effect of various types of instruc- 
tions on tbe rate of acquisition and extinction of the conditioned lid re- 
sponse bas been studied systematically by J. Miller (1939). Some of bis 
results under different instructions are summarized in Table 8. Efforts 
to inhibit response (Groups 1 and 2) reduce tbe frequency very little 


TABLE 8 


Effect of Supplementary Instructions on Frequency 
of Conditioned Responses 


Conditioned stimulus, light; unconditioned stimulus, air puff to one eye; interval 400 
msec. The groups are arranged in order of increasing success of conditionins? 
(Modified from J. Miller, 1939) S ‘ 


Group 


Per Cent of Trials 
on Which Conditioned 
Instructions Responses Appeared 


(1) Inhibitory 


(2) Voluntary 
antagonism 

(3) Noncommittal 

(4) Informed 

(5) Facilitatory 


“Be sure that you do not wink or start 
to wink before you have felt the -puff “ 
(n — 20) 

“Open your eyes each time the air puff 
strikes your eye’’ (n = 20) 

Only minimum instructions necessary for 
photographic recording (n = 25) 

Told that light would be followed by 
n case yon feel your eyes closing or 
air puff (n = 20) 

^ V° se ’ n °thing to prevent 

it (n = 20) 


26 

28 

38 

44 

71 




CLASSICAL AND INSTRUMENTAL COMPARED 205 

below that of the control groups unmstrueted with respect to response 
(Groups 3 and 4) This suggests that the subjects in Groups 3 and 4 
may ha\e adopted an inhibitory set without specific instructions Instruc 
tions not to prevent responses (Group 5) resulted in a greatly increased 
frequency of response, and m slow extinction The form of the responses 
showed clearly that they were not voluntary reactions The results eon 
firm the conjecture that conditioned eyelid responses normally de\elop 
against opposed inhibitory sets (Hilgard, 1938) 

The conclusion that inhibitory instructions do not reduce the level of 
conditioning much has been questioned by Norris and Grant (1948) who 
believed that the responses counted as CR’s in this study were actually 
beta responses They repeated the study using conditions essentially like 
those of Groups 1 and 3 of Miller’s study Their results shown in Figure 
4 12 indicate that there is a very large difference between the levels of 



FIGURE 1 12 The effect of instructions upon the level of conditioning The 
emphases of the two d./Terent sot. of ' .ojIrMMM M) 

tions appearing w.th.n the body of the figure (Nonas and Cr.nt, IMS) 

cond.tton.rtg obta.ned nndcr rnhtb.toty and noncommittal ntstmcttona 
In any event, it is ev.dent that the natnro of the mstruct.ons can drier- 
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mine the degree of conditioning, even though the response is not com- 
pletely under voluntary control. 

The degree to which specific instructions can influence the conditioned 
eyelid response has been further studied in the discrimination experi- 
ment. In a study of the effect of instructions, all subjects were condi- 
tioned on the first day to the positive stimulus, all trials being reinforced. 
On the following two days, during which the negative stimulus was intro- 
duced without reinforcement, the groups differed in the instructions 
given. The resulting characteristics of responses to the positive stimulus 
are summarized in Table 9. 


TABLE 9 

Effect of Instructions on Characteristics of Conditioned Eyelid 
Responses to Positive Stimulus Within a Discrimination Experiment 
(After Hilgard and Humphreys, 1938a, p. 301) 

Groups in Order Characteristics of Responses to 

of Amount of Voluntary Positive Stimulus 

Supplementation of Response 


10 subjects in each group 

FREQUENCY 
PER CENT 

AMPLITUDE 

Mil. 

LATENCY 

MSEC. 

Instructed to refrain from responding 
to either stimulns 

55 

13 

448 

Instructed to refrain from responding 
to positive stimulus hut instructed to 
respond to the negative stimulus . 

71 

16 

404 

Without instructions with respect to 
response 

74 

19 

401 

Instructed to respond to positive stimu- 
lus, not to negative stimulus . . . 

90 

35 

319 


Responses are found to show increasing frequency, increasing ampli- 
tude, and decreasing latency when they are arranged in order from 
voluntary restraint through no instructions to voluntary facilitation. 
This is evidence that instructions make some control over the process 
possible. On the other hand, voluntary restraint does not succeed com- 
pletely in overcoming the positive conditioning tendencies. Corresponding 
results were found for responses to the negative stimulus. Except under 
direct instructions to respond, the negative stimulus always elicited re 
spouses at lower frequency and with smaller amplitude than the positive 
stimulus, but the same trends were found. Conditioned responses per 

S f a m , SP f e ° £ TOlUntaiT effort t0 ‘tan, ana they increld 

when voluntary restramt was reduced as well as when there was adTed 
voluntary effort to respond. auciea 
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SUMMARY 

The distinction between classical and instrumental conditioning on 
operational grounds naturally leads to the question whether there are 
other differences between these two forms of learning and, for that mat- 
ter, whether they are even forms of the same basic process One conceiv 
able way of attempting to obtain an answer to this question would be to 
condition the same response mstrumentally and classically, and to com 
pate the progress of learning under these different circumstances Such 
experiments have not been successfully carried out, however, for several 
reasons (1) It is impossible to perform a purely instrumental condition 
xng experiment because such learning requires reinforcement (in the 
weaker sense in which that term is used) Since reinforcement must 
occur in some situation, there is always a chance for classical condition 
mg to occur, and for the responses to reinforcement to be conditioned 
to these situational cues (2) It is also very difficult to perform a strictly 
classical conditioning experiment because the classical CR usually has 
some effect upon the stimulation al consequences of the UCS (3) When 
tile two forms of conditioning have been compared the actual responses 
occurring under the two procedures seem always to have been somewhat 
different This effect may be unavoidable (4) There is good evidence that 
most (we think all) responses which can be conditioned classically cannot 
be conditioned mstrumentally, and vice versa 

On the other hand, classical conditioning and instrumental condition 
mg possess many features m common They seem to respond similarly to 
manipulations of many experimental variables The only important ex 
ception is intermittent reinforcement which is more detrimental to classi 
cal than to instrumental conditioning In this chapter we have described 
fifteen basic phenomena of conditioning which classical and instrumental 
conditioning both display The fact that the two forms of learning are 
similar in so many respects means one of two things Either the in 
escapable classical conditioning component of all learning is responsible 
for these effects or classical and instrumental conditioning really arc 
different forms of the same basic process There is to our knowledge, no 
psychologist who has taken the first of these positions 

If classical and instrumental conditioning are indeed two different 
forms of learning process the next question to arise is whether thej 
should be considcied as separate at all Some psychologists such as Hull 
and Guthrie have maintained that distinction cannot be made at other 
than the operational level but the majority have adopted two process 
theories of one sort or another, maintaining that a distinction is neccs 
sary At this level, distinctions between classical and instrumental con 
ditiomng have involved two general ideas It has been held (1) that the 
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laws of learning which apply to the two procedures are somewhat differ- 
ent and (2) that the responses which can be manipulated by these two 
mechanisms are different. As a general thing, the two-process theorists 
have held both of these hypotheses at the same time. It has seemed 
desirable to postpone discussion of the first of these propositions to a 
later chapter. It has been noted, however, that the most common form 
of the two-process view holds that classical conditioning is learning by 
contiguity and that instrumental learning is governed by the law of 
effect. Anticipating later discussion, we may say that the evidence with 
regard to this theory is more strongly positive than negative. 

When we turn to the second idea, we find that it is possible to identify 
two separate positions regarding the responses which have been consid- 
ered to be manipulable by classical and instrumental conditioning. One 
view is that classical conditioning is stimulus-stimulus learning, which 
serves to establish preparatory reactions, whereas instrumental condi- 
tioning is stimulus-response learning. The alternative position is that 
both forms of learning can be conceptualized within the S-R framework, 
but that some responses can be conditioned classically, and other responses 
can be conditioned only instrumentally. 

The evidence favoring the last of these points is of the following sort : 
There seem to be no established demonstrations that exactly the same 
responses can be conditioned both classically and instrumentally: (1) 
Autonomic responses apparently cannot be instrumentally conditioned 
at all. (2) Responses, such as the eyeblink, which seem to be modifiable 
both classically and instrumentally, turn out upon close inspection to be 
different responses in the two eases. And (3) the voluntary control over 
autonomic reaction which suggests that instrumental conditioning of 
these responses might be possible probably always requires a mediating 
response involving the central nervous system. 

Although such evidence leads quite directly to the conclusion that 
classically and instrumentally conditionable responses correspond to 
what we know in personal experience as involuntary and voluntary re- 
sponses, this does not mean that voluntary factors have no influence over 
classical conditioning. There is good evidence that they do, and it seems 
quite likely that, in the actual case, the conditioned response is a eom- 
binatmn of voluntary and involuntary processes. The same state of affairs 
probably also exists in the case of instrumental conditioning One of the 
most difficult tasks in the psychology of learning is to unravel these 
components so that they can he dealt with separately. 
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Practice and the Strength 
of Conditioning 

One op the most basic questions in the fields of conditioning and learn 
mg concerns the form of the function which relate* these processes to 
practice This question has been a matter of discussion almost from the 
beginning of the study of learning It has been proposed by some that 
learning is a sudden insightful process bj others that it occurs gra du 
ally, and by still others that it occurs suddenly in some situations and 
gradually in others In spite of extensive argument the issue has not 
yet been resolved The reasons for this are to be found in a set of difficult 
methodological problems which must be considered before turning to a 
presentation of the mam theoretical positions regaidmg the relationship 
between learning and practice 

METHODOLOGICAL PROBLEMS 

In the final analysis the major methodological difficulties confronting 
the psychology of learning stem from this fact the psychologist who 
sets out to discover the laws of learning must rely in his search upon 
indirect manifestations of the process in which he is interested What 
is involved here is the learning performance distinction developed earlier 
(p 4) Although the psychologists interest is m learning all of the 
data available for analjsis are performance data The problem of assess 
ing the value of the former from measures reflecting the latter is a very 
difficult one 

The Problem of Response Specification 
The difficulties begin at the basic level of response specification Most 
students of learning following Skinner (1938) define a response m terms 
of its consequences as any act on the part of the organism which has 
some particular effect upon the environment Such definitions are in 
terms of the requirements of the experiment onJj There is no specific 
loa 
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concern for the details of the movements which press the bar, lift the limb 
in response to a bnzzer, or transport the rat through the maze. Yet, it is 
known that the exact form of these movements changes with practice. 
This creates a question of the legitimacy of combining the various forms 
of a reaction to produce a learning curve. Is it justifiable, for example, to 
average amplitudes or latencies of responses as if they represented the 
dimensions of some single reaction when it is known that the precise re- 
sponse changes with practice? 

However indefensible this procedure may appear, the alternative has 
seemed even less appealing to the majority of psychologists interested in 
learning. This alternative is to regard quantitatively different reactions 
as if they were qualitatively different ; that is, to treat an eyeblink with 
a latency of 300 msec, as different from one with a latency of 450 msec, 
in somewhat the same sense as either of them is different from a patellar 
reflex or a salivary response. Although this view of the response is obvi- 
ously cumbersome, there is some evidence to recommend it. This evidence 
consists in a variety of demonstrations that the organism can be taught 
to give responses which have certain quantitative values, in much the 
same way that it can be taught responses with specified topographic char- 
acteristics. Eerster and Skinner (1957), for example, show that pigeons 
can learn to respond slowly or rapidly if reinforcement is made con- 
tingent upon these aspects of behavior. Logan (1956) has attempted to 
develop a theory of learning which starts with the assumption that dif- 
ferent response values qualify as different responses. He calls it a micro- 
molar theory of learning. The choice between this position and the more 
obvious conventional one depends, finally, upon which is more successful 
in expressing lawful regularities in the field. 


Psychometric Problems 

The traditional viewpoint, that variations in the different measures 
reveal differences in the strength of a single response entity, is not with- 
out problems of its own. Specifically, it raises a series of questions about 
the value of the various indices as measuring instruments. Of such ques- 
tions, the first to be mentioned concerns reliability. Probably because of 
the relative simplicity of the conditioning situation, the reliability coeffi- 
ciente have tended to be satisfactory. Thus, Schlosberg (1932) reports a 
reliability coefficient of +.66 for the amplitude of the conditioned knee 
jerk, and Grant and Schneider (1949) report one of +.92 for the GSR 
A second problem of this general sort, associated with measures of eon- 
chtioning, invdves their ability to reflect changes in conditioning at a ff 
levels. Probability measures suffer from the fact that they have a°ceilin- 

llMTlttZTZ'" niKler “ C ° nditi0ns >>oti reach 100 
per cent, although other measures (such as resistance to extinction) trill 
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reflect differences Sifflilariy, measures of speed and vigor are restricted 
in range by the capacities of tile subject and cannot reflect increases in 
learning beyond this physiological limit. 

A more difficult problem concerns the extent to which different meas- 
ures reflect the same thing There is a heterogeneous body of evidence to 
indicate that there are important differences among them, and that some 
measures are influenced more than others by various experimental pro- 
cedures. (1) Studies of the effect of motivation on learning (Hillman, 
Hunter, and Kimble, 1953) have shown that running speed in the maze 
changes markedly with changes in drive level but that number of errors 
does not. (2) Estes (1944) reports that punishment during extinction 
slows down the rat's rate of responding in the Skinner box but has no 
effect upon resistance to extinction . (3) The reliabilities ol frequency and 
amplitude measures have been found to be appreciably higher than those 
of latency measures. (4) Reinforcements beyond the first in the Skinner 
box sometimes produce no change m rate of responding but do increase 
numbers of responses extinction (Skinner, 1938). (5) Partial rein- 
forcemeat scheduling has a more profound effect upon resistance to ex- 
tinction than it does upon percentage of conditioned responses during 
training (6) When they have been compared, the correlations among 
various measures have usually been too low to support a common-process 
view. Table 10 presents some of the relevant data It will be seen that the 
majority of the correlations indicate relatively poor correspondence 
among the various measures Indeed, when the correlation is at all sub- 
stantial, it sometimes turns out that the measures being correlated are 


TABLE 10 

Correlations Among Various Response Measures 
Obtained in Conditioning Experiments 


Investigator (s) 

and Response 

Measures Correlated 

Correlation 

Kellogg and E L 

Frequency and amplitude 


Walker (1938b) 

Latency and amplitude 


Flexion (dog) 

Latency and frequency 


A. A, Campbell and 

Frequency and amplitude 

+ C3 

Hilgard (193G) 

Latency and amplitude 

-5i 

Eyebliuk (man) 

Latency and frequency 

A. A. Campbell 

Frequency and amplitude 

-(-63 

(1938) 

Latency and amplitude 


Knee jerk (man) 

Latency and frequency 

Frequency and resistance 1 o extinction 

-60 

Hunter (1935a) 

Banning response 
based on shock (rata) 
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not separate measures at all. The typically moderate correlations between 
percentage of conditioned responses and amplitude, for example, often 
occur because the amplitude measure includes instances of zero amplitude, 
cases in which no response occurred. Thus the two measures are not in- 
dependent, because amplitude is, in part, a frequency measure. When 
only amplitude of conditioned responses is counted, the correlations are 

markedly reduced. 

The suggestion that the different measures reflect different processes 
carries with it the implication that some measures may be better than 
others as indicators of learning, and this raises the question of which 
indicator is best. At the present time no definite or final answer to this 
question seems possible. Different measures are appropriate in different 
situations, and a variety of practical considerations helps to determine 
which to choose in connection with a given experimental problem. For 
example, resistance to extinction is not used as a measure of learning very 
often, because it is so costly in terms of experimental time and subjects, 
especially where one’s purpose is to trace the acquisition function. Other 
measures provide an index of the strength of conditioning on every trial ; 
but to obtain comparable information using resistance to extinction as 
the measure, it would be necessary to condition and extinguish a separate 
group of subjects for each trial point. Along the same practical lines, rate 
of responding is an easy measure to obtain in free-responding situation, 
time scores and error measures lend themselves to studies of maze learn- 


ing, and percentages of correct responses are convenient to use in experi- 
ments involving blocks of trials. 

Another set of considerations, also largely practical, which determines 
the choice of a measure, is statistical in nature. Response speed, for ex- 
ample, is often a better measure than response latency for such reasons. 


Latency measures tend to be badly skewed in the positive direction, since 
very long latencies are possible and negative ones are not. As a result, the 
means of latency measures are difficult to compare by conventional sta- 
tistical methods. Converting to a speed measure (reciprocal latency), 
however, tends to normalize the distribution. Other transformations, such 
as the logarithmic one for time scores and the arcsin transformation of 


percentages, are also used for the purpose of normalizing distributions 
and making them amenable to statistical treatment (Mueller, 1950). 

Finally, at a more abstract level, there are certain implicit initial as- 
sumptions made by all learning theorists which suggest the selection of 
one measure rather than another in a given situation. Of such assump- 
tmns the commonest is that learning increases uniformly with practice 
and that for purposes of depicting the acquisition ' process, a measure 
winch changes monotonically with practice is better than one which 

C t !f” a noiunon °lonic manner. Thus, although latency and other 
time-based measures are useful for plotting learning curves in many in- 
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strumental situates (Graham and Gagne, 1940), they are less often 
employed in classical conditioning situations where they may decrease at 
first and then increase (Boneau, 1958) Similarly, response magnitudes 
were the standard measures in Pavlovian conditioning, but are less often 
used m instrumental conditioning because they either change very little 
m the course of conditioning or tend to come to match the minimal value 
required to obtain reinforcement Thus, if it takes 20 gm of pressure on 
the bar to get food, the rat may press with increasing force for a time, 
but gradually the pressure will come to approximate 20 gm (Hull, 1943, 
pp 305 306) It should be recognized that even monotonically increasing 
measures probably do not reveal the underlying processes directly At 
tempts to develop rational measures which truly reflect learning have not 
met with much success 1 


Typical Learning Curves 


The different measures of learning behave in different ways with prac 
tice Amplitude, probability of occurrence speed of responding, and rate 
curves show an increase, latencies and other time measures decrease 
Probability and percentage of response curves often show a double in 
flection Since conditioned responses sometimes do not appear until after 
several reinforcements, the first portion of the curve may be flat This 
portion is followed by a positively accelerated increase, which is soon re 
placed by a negatively accelerated one, as a maximum is approached The 
exact course of conditioning revealed by this measure varies considerably 
depending upon the level at which the function begins Curves which 
start at a low level show a more protracted period of positive acceleration 
than those with a higher starting point 

Resistance to extinction has been used so rarely as a measure of condi 
honing as to make firm statements of the function by which it reflects 
learning somewhat risky There is however, unanimity among the data 
available For eyelid conditioning (Kimble and Dufort, 1956, Prokasy, 
1958) and for bar pressing (Williams, 1938, Penn 1942, Miles, 1956) 
the functions have been negatively accelerated and increasing 

Skinner and his associates (Skinner, 1938, Ferster and Skinner, 1957) 
have reported the results of their studies almost exclusively m terms of a 
cumulative response measure A set of typical conditioning functions of 
this sort appears in Figure 51 This curve differs from the ones in the 


1 See, for example, the senes of papers on the quantification of "action potential hj 
Hull and his students (Felsmger, Gladstone, Fawaguchi, and Hull, lOiJ. Qiadmnc, 
Yamaguchi, Hull, and Fclsinger, 1947, Hull, Felsmger, Gladstone J « 

1947) and the comment by Koch (1924) Spence has sometimes used what amounts ta 
a Z score transformation to obtain reaction potential from rw^nw prolwbilitlrt, but 
he has also claimed that this procedure is, at best, a poor first approximation (Spence, 


1958b) 
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FIGURE 5.1. Some typical response curves obtained from a bar-pressing ex- 
periment with rats. The curves are tracings from bar-pressing records of 
individual rats. After Skinner (1938) as presented by Keller and Schoenfeld 
(1950). 

preceding examples in that the base line is time rather than trials and the 
ordinate is cumulative. Because of this latter feature such functions do 
not flatten with protracted practice, and it is difficult to tell the degree to 
which such learning functions reflect the same process as the more com- 
mon ones. Youtz (1938a) has plotted data obtained by Skinner’s method 
in somewhat more conventional units. He finds that the time required for 
10 successive responses in the lever situation gradually decreases in a 
curve roughly resembling that of Thorndike’s time-reduction curves for 
trial-and-error learning. The curve cannot be carried beyond the first 30 
or 40 reinforcements in this manner, because the ingestion of food changes 
the drive, and the “learning curve” becomes more nearly a “satiation 
curve.” 


The Problem of Averaging 

Most learning curves are for groups of subjects, rather than individ- 
ual For many purposes this creates a problem, especially in experiments 

TtrT 7? • T 7-l° me cnterion s ™h as 100 per cent condition- 
, ? , bluck of trials. Different subjects will take different numbers of 

trials to reach the criterion, and it becomes difficult to find a base line 
against which to plot the response measures to represent the course of 
acquisition. One solution to this problem is in +ho nm T 7 -- 

(Vincent, 1912; Kjecstad, 1919). The tota! 

L d a aed in ‘“ : fraCti0Ml ***• «• 

<tuiri„ g 35 trials the ^ s toji a b e 3 ^ \ "*** 

10 trims, they wonld he 4.0 trials long, and so^LfhlteSdTata 



116 


CONDITIONING AND LEARNING 


tion is possible though not common, in the averaging ol stepyise curves 
Tr uck go from 0 to 100 per cent in a single trial. Figure 5.3 illustrates 
rvhat can theoretically happen under these circumstances. In actual eu- 
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Campbell (1936) and Spence (1956) have presented data for homogene- 
oils subgroups of subjects. Skinner relies almost exclusively upon the 
subjects to illustrate various influences upon tie progress 


Tlie Learning-Performance Problem 

Most theorists either assert or imply that the level of learning, itself, 
provides a hypothetical upper limit which performance would reflect 
directly under absolutely ideal conditions. Unfortunately, however, con- 
ditions never are completely optimal; and performance, therefore, always 
underestimates the level of learning to a degree which varies directly 
with the extent to which the training conditions depart from the ideal. 
To mention just one specific example, the mere act of responding is known 
to create an inhibitory state which interferes with performance. For 
reasons of which this example is typical, it is very difficult to isolate the 
learning process and to study it alone in any given situation. 

In a general way the nature of the solution to this problem is obvious. 
It is necessary, first, to discover the variables which influence perform- 
ance, to determine the laws by which they lower performance, and then, 
by the extrapolation of such functions, to determine the level of perform- 
ance which would be obtained with all performance variables set at their 
optimal values. Such levels plotted as a function of practice would reveal 
the progress of learning in a given situation and would constitute the 
learning curve for that situation. In actual practice, this general program 
is very difficult to implement for a variety of reasons: (1) It is probably 
overly optimistic to hope that the factors involved will divide themselves 
co-operatively into learning and performance variables. Many will, no 
doubt, turn out to be both. The amount of reward given a rat for run- 
ning a maze, for example, may very well turn out to have an influence 
directly upon habit structures, as well as to have a performance effect 
through a motivational (incentive) mechanism. (2) Different perform- 
ance variables will probably be of different degrees of importance in 
different learning situations. Again, to be specific, the temporal spacing 
of practice trials is quite important in human motor learning where large 
amounts of work are involved and relatively less important in human 
verbal learning or eyelid conditioning which require relatively effortless 
responses. Such considerations mean that it is impossible to generalize the 
function obtained in one situation to other, different, situations with any 
degree of confidence. (3) Different response measures seem to respond 
differently to different variables. There will be numerous occas.ons in 
what follows to call attention to one of the important applications of this 
point: Motivation has a marked effect upon speed ami vigor measures, 
little or none upon measures reflecting only the accuracy of performance. 
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Campbell (1936) and Spence (1956) have presented data for homogene 
ous subgroups of subjects Skinner relies almost exclusively upon the 
curves of single subjects to illustrate various influences upon the progress 
of learning 
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(4) The methodological techniques for separating the learning and per- 
formance variables have not been worked out. Analysis-of-vanance ( ae- 
torial design) methods (see below) which are most often used are o 
limited applicability; they are completely satisfactory only m eases where 
the evidence points unequivocally to the conclusion that the vanab e 
under investigation is a performance variable. 


Separation of Learning and Performance by Factorially 
Designed Experiments 

Practically speaking, a major difficulty encountered in separating the 
learning and performance variables is that of establishing criteria by 
which they may be distinguished. The criterion most commonly accepted 
for this purpose derives directly from the definition of learning as a rela- 
tively permanent change in behavior which occurs as a result of practice. 
Thus changes which occur without practice (as a result of deprivation, 
feeding, or rest) are taken to be performance changes. In many cases, 
however, it is impossible to change the value of the variable in which we 
are interested without also giving the subject training. If we want to 
study the effect of a change from immediate to delayed reinforcement, 
from continuous to partial reinforcement, or from massed to distributed 
practice, it is necessary to provide training in order to allow the changes 
to produce their effects. For problems of this sort, the factorially designed 
experiment is commonly accepted as a method of separating learning 
effects from performance effects. 

In experiments of this sort, two or more groups of subjects are first 
trained under different conditions, known to produce different levels of 
performance. Then, each of the original groups is subdivided. One sub- 
group continues under the original condition; the other is switched to 
the alternative condition. The decision as to whether a variable influences 
learning or performance depends upon what happens in the second phase 
of the experiment, after the shift. If the change in conditions alters the 
subject’s performance appropriately, leaving no indication of an effect 
of the previous training condition, the variable is assumed to be a per- 
formance variable. If the previous training has a residual effect, even 
after the training conditions are altered, the variable is assumed to be a 
learning variable, at least in part. Examples of the application of this 
method to testing the effect of CS intensity, motivation, and amount of 
reinforcement will help to make the method clear. 

1. Conditioned Stimulus Intensity. Razran (1957a) cites a considerable 
amount of Russian literature to show that conditioned stimulus intensity 
has a marked effect upon response strength. As is usual with such evi- 
dence, this leaves open the question whether the influence of CS intensity 
is on learning or performance, since no special procedures were imple- 
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mealed for separating the two processes Tlie factorial method of deter- 
mining whether the effect of CS intensity is on learning or on perform- 
ance is nicely illustrated by the studies of this problem by Grant and 
Schneider (1948, 1949). In one of their studies, these investigators con- 
ditioned the GSR’s of four groups of human subjects to tones of four 
different intensities. Then, for extinction, they subdivided these groups 
into four smaller groups, making sixteen groups in all. In the extinction 
procedure, each subgroup was tested with one of the original tones. The 
logic of the analysis used to evaluate the results of factorial experiments 
can be made clear by reference to the results of this experiment in Table 
11. In this table, the rows represent the four conditioning intensities, 
from greatest to least. The columns represent extinction intensities, from 
least to greatest. The whole table illustrates the experimental design, in 
which subjects were conditioned under one procedure and then sub- 
divided for extinction. 

It is essential, in interpreting these data, to remember that the entries 
in the table are GSR magnitudes during extinction Differences among the 
row means (165 8, 275 8, and so on) reveal the effects, if any, of the in- 
tensity of the CS during conditioning , since these means are based on 
subjects originally trained under varying conditions, with CS intensity 
equated during extinction The column means, on the other hand, reveal 
the effects of the intensity of the CS during extinction, since as a group 
the subjects whose performances these means reflect have had identical 
histories In learning-performance terms the row means are learning 
measures , the column means are performance measures 

Looking first at the row means, it is clear that the differences among 
them present no consistent pattern The lowest GSR magnitude occurs to 
the strongest stimulus and the greatest GSR magnitude is in response to 
the next to-the-strongest stimulus Appropriate statistical analysis sup- 
ports this impression there is no significant effect of CS intensity upon 
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learning. Tlie col umn means do show a trend in the expected direction, 
but it is not of sufficient magnitude to satisfy the statistical requirements 
for signific an ce. Thus we come to the conclusion that CS intensity has no 
effect upon learning and that its effect upon performance, described by 
Razran, has not been verified. Studies by Carter (1941) using eyelid con- 
ditioning procedures and by Hovland (1937b, 1937c, 1937d) again with 
the GSR lead to the same conclusion. 

One possible complication in all of these studies is that they have used 
human subjects, whereas the positive results from the Russian labora- 
tories were obtained with animal subjects. It may be that this accounts 
for the differences in results. Specifically, the conditioned responses of 
human subjects may be mediated by subvocal verbal processes which, in 
a sense, equate the CS intensities for all groups. That is, when the CS 
comes on, the subject may respond implicitly with some reaction such as, 
“there it is,” or “the tone,” which would tend to make the situation 
similar for all subjects regardless of the intensity of the CS. On the 
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Recently Perkins (1953) has argued that the effect of CS intensity is 
to he explained in terms of a contrast phenomenon. His point is that 
learning to respond to any stimulus, whatever its intensity, always in- 
volves the m akin g of a discrimination, between the situation with the CS 
present, and the situation with the CS absent. Specifically, what the ani- 
mal must learn is to respond to the former condition and not to the latter. 
Now. if the animal is tested with a series of stimuli of varying intensities, 
it is to be noted that the presentation of a weak stimulus resembles the 
stimulus-absent condition, under which the subject has learned not to 
respond. Conceivably the failure to respond would generalize to this 
condition, since it is known that extinction effects do generalize. At least 
up to a point, stimuli of increasing strength would resemble the positive 
training situation more and more, with the result that there would be an 
increasing tendency for the CR to occur. This hypothesis, thus, explains 
the stimulus intensity phenomenon in what are essentially learning terms. 
Experiments performed to test the theory, by and large, have been posi- 
tive (Perkins, 1953; Bragiel and Perkins, 1954; Johnsgard, 1957). 
"Whether the entire stimulus-intensity effect can be so explained remains 
to be seen. 

2. Motivation. The effect of motivation upon performance has been the 
subject of a large amount of research which will be described in more 
detail in Chapter 13. For our present purposes, it provides a particularly 
clear example of the method of separating learning from performance 
variables by the factorial experiment. The study of Hillman, Hunter, and 
Kimble (1953) is typical. Two groups of rats were trained on a complex 
maze for a water reward, initially under 2 and 22 hours of water depriva- 
tion. After 15 initial training trials, the groups were subdivided. Half of 
the animals in each group continued under the original deprivation con- 
dition ; the other half were switched to the alternative condition. The re- 
sults in terms of running time appear in Figure 5.6. The important point 
is that the snitch nr deprivation produced a rapid shift in performance, 
to the level characteristic of the group which had been on the same drive 
from the beginning. 


3. Amount of Reinforcement. Several experiments, employing both 
classical and instrumental conditioning procedures, have shown that the 
amount of reinforcement influences performance in the learning situa- 
tion. Passey (1948) conditioned the eyelid responses of human subjects 
using air puffs of four different pressures as the unconditioned stimuli 
and obtained better conditioning with the stronger puffs. Spence (1956) 
reports several other demonstrations of the same effect. Crespi (19441 and 
Zeaman (1949a, 1949b) have both reported studies which show that rats 
traverse a straight runway more rapidly for a large amount of food than 
ror a small amount of food. Guttman (1953) manipulated the quality of 
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FIGURE 5 6 The results of a factonally designed experiment to determine 
the effect of drive and drive-shift upon the speed with which rats rim a maze 
For 10 trials two groups were run under 2 or 22 hours of food deprivation 
From Trial 11 on, the original groups were subdivided Two groups (2 2, 
22 22) continued under the original motivational condition Two others 
(22 2, 2 22) were switched to the alternate condition The important point is 
that performance on Trials 11 IS seems to depend only upon motivational 
level during these trials, that is, drive affects only performance (Hillman, 
Hunter, and Kimble, 1953) 

reinforcement for rats in a Skinner box m a different way, by using liquid 
reinforcements of different sweetnesses He showed tint both performance 
under a periodic reinforcement schedule (p ICO) and resistance to ex 
function varied with sweetness These data leave no doubt that perform 
ance m the learning situation depends upon the amount of i deforcement 
There is now rather good evidence that the effect is upon performance 
The evidence comes from studies m which the amount of reinforcement 
was increased or decreased during the course of the experiment Crespi 
and Zeaman included this feature as a part of their studies, and both 
obtained rather abrupt changes in speed of running 7eaman's data pre 
sented graphically in Figure 5 7 show that changes in amount of rein- 
forcement produce a sudden increase or decrease in response latency in a 
way which parallels that produced by changes in motivation Tins is the 
sort of result, to be expected with a performance variable 
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TRIALS WITH AMOUNT OF FOOD REWARD REVERSED 

FIGURE 5.7. The effect of a change in food reward upon the latency of the 
response in the Graham-Gagne runway. The acquisition asymptote is esti- 
mated from latencies on the last several trials before the shift in amount of 
reward. Note that the change in amount of food employed as a reinforcer 
produces an immediate shift on trial 2 after one trial with the new amount. 
Adapted from Zeaman (1949). 

Evaluation of tlie Factorial Procedure 

From the evidence just cited it is clear that the factorial method can 
he a powerful tool in determining the place at which a particular variable 
has its effect. This is true whenever the results of an experiment are 
clearly of the sort to be expected when the influence is entirely upon 
performance. In other eases, the data may be much more difficult to in- 
terpret. To illustrate the point, suppose that the curves in Figure 5.8 
represent the outcome of a factorially designed experiment intended to 
determine whether a certain variable influences learning or performance. 
With the change in conditions, performance also changes, but only gradu- 
ally, and it is several trials before the curves for the groups subjected to 
the change reach the levels of the groups which had not been switched. 
Do these results show an effect upon learning, performance, or both? In 
the statistical procedure employed in the factorial experiment, the answer 
depends upon the portion of the performance function selected for evalu- 
ation. If the section of the function immediately after the shift is selected, 
the result will be one which seems to indicate that the variable influences 
both learning and performance. Spence’s (1953) assignment of both roles 
to the UCS intensity variable apparently was on the basis of data of this 
sort. If the last portion is selected for analysis the effect will be on per- 
formance. It, thus, becomes clear that the value of the factorial method 
is more limited than is sometimes believed ; it finds a straightforward 
application only in the case where changes in experimental procedure 
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FIGURE 5 8 Graph showing the outcome of a hypothetical factonally dc 
signed experiment which would be difficult to interpret The terms high and 
low refer to the magnitudes of the variable manipulated If statistical analyses 
are earned out on data in section A of the post shift graph, the effect will 
appear to be upon learning and performance If the data in section B are 
evaluated, the effect will appear to be only upon performance 

produce immediate shifts to the level of performance typical of subjects 
trained from the beginning under the new condition 


Other Performance Variables 

The set of performance variables considered above does not exhaust the 
list of conditions which appaiently affect behavior at the performance 
level Two other sets of performance variables should be mentioned Of 
these the temporal distribution of practice trials is the most important 

The typical effect of the crowding of practice trials m time (massed 
practice) is to diminish performance to a degree which depends upon the 
degree of massing (Calvin, 1939, Reynolds 1945, Vandermcer and 
Amsel, 1952 , Spence and Norris 1950) The results of the last mentioned 
study appear m Figure 5 9 Theoretically this diminution has been at 
tnbuted by the Pavlovians to inhibition m th i cxnforccmcnt, by Razrau 
(e g , 1930, 1955c) to overrent forcement, and by Ilovland (193G) to mli»- 
bihon of remfoi cement The evidence that what is involved is a perform 
ance factor comes from a variety of demonstrations that the effect of 
massed practice is temporary and that the decrements! process dissipates 
spontaneously with rest Reminiscence in verbal and motor learning and 
spontaneous recovery following extinction have both been interpreted m 
this way In conditioning experiments similar evidence sometimes is ob- 
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FIGURE 5.9. The effect of the time between trials upon the acquisition func- 
tion for eyelid conditioning (Spence and Norris, 1950). 

tained in a distortion of the form of the extinction cnrve. These functions 
tend to be decelerated, response reduction being rapid at first and then 
gradually tapering oft as the base line is approached. Following massed 
practice, however, extinction curves occasionally show an initial rise, as 
if they had previously been depressed by some inhibitory mechanism. 
Hovland (1936) conjectured that the inhibitory process was an “inhibi- 
tion of reinforcement” and that the omission of reinforcement at the be- 
ginning of extinction changed the stimulus situation enough to produce 
disinhibition. On these assumptions he predicted that the usual extinction 
curve would occur after spaced practice, the rising type with massed 
practice. Accordingly he conditioned the GSR ’s of human subjects under 
massed and distributed practice and then extinguished them. His results 
as shown in Figures 5.10 and 5.11 agreed with his prediction. It is im- 
portant to note that the rising type of curve could be obtained only if 
training were under conditions of massing unci if extinction began imme- 
diately. Postponing extinction would produce the monotonic decreasing 
type of function even if conditioning occurred under conditions of mass- 
ioff- This suggests that the lapse of tune allowed the inhi bition to dissipate 
and that no initial rise was possible. 
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FIGURE 5 10 Extinction of galvanic 6kin response immediately following 24 
distributed reinforcements Mean, 20 adult human subjects {Hovland, 1936) 
FIGURE 5 11 Extinction of galvanic skin response immediately following 24 
massed reinforcements Mean, 20 subjects Note the initial rise If reinforce- 
ments were fewer or if time elapsed before extinction began, this initial rise 
was lacking (Hovland, 1936) 

The other additional set of performance factors derives from the con 
sideration that most learning is complex and requires the simultaneous 
learning of several components Possibly the clearest illustration of this 
is in the appearance during learning of plateaus (Bryan and Harter, 
1897) during which practice does not lead to improvement In complex 
learning plateaus often seem to be temporary periods devoted to the 
organization of component habits into larger units or to the learning of 
particularly difficult portions of a problem (Hunter, 1934) In code 
sending, for example plateaus may occur when the learner is making the 
transition from a letter habit to a word habit Although this transition 
depends upon learning there is no concurrent improvement Even at 
simpler levels of learning similar complexities arise For example the rat 
m the Skinner box learns to press the bar only as competing reactions are 
extinguished (Estes, 1950b) It also seems quite likely that the more posi 
tive components of what the animal learns comprise at least two habits 
One of them involves conditioning to elements m the experimental situa 
tion of responses to food so that animal learns that food is available The 
other consists in the acquisition of the bar pressing habit It seems un 
likely that these two processes would develop as exactly the same function 
of practice or that they would be reflected equally well by any given 
measure of performance 



128 


CONDITIONING AND LEARNING 


the nature of the acquisition function 

If ive liad a complete understanding of the influence of all of the per- 
formanee variables upon the progress of behavioral change in the learn- 
ing situation, it would be much easier to determine whether a single ac- 
quisition function applies to all learning, and if so, the nature of that 
function. We do not have this information, however, with the result that 
alternative theories about the nature of the acquisition function have 
been offered. Two hypotheses have been advanced: (1) Some theorists 
have held that learning is a gradual process, as the learning curves sug- 
gest. (2) Others believe that learning occurs in a single training trial, but 
that factors in the performance situation somehow obscure it. The first of 
these theories is called a continuity view of learning. The second, one-trial 
position is a noncontinuity view. In the disputes which have arisen over 
the issue implicit in this difference of opinion, the burden of proof seems 
to rest with the theorist accepting the one-trial view, because the per- 
formance functions obtained in almost all learning experiments show a 
gradual course of acquisition. Clearly the easiest assumption to make is 
that the underlying process is also gradual. This view has been most 
vigorously supported by Hull (1943) and Spence (1945). 

Opposed to the continuity position are two discriminable variations of 
the noncontinuity view. One of these was originally Guthrie’s (1935) 
position. Its more recent champions have been Estes (1950), Sheffield 
(1949), and Voeks (1950). The other position derives from the writings 
of Lashley (1929) and Krechevsky (1932a, 1932b, 1933a, 1933b). The 
basic difference between these two versions of noncontinuity theory has 
to do with the question of what is learned. The followers of Guthrie hold 
that it is S-R connections ; those in the Lashley tradition view learning 
from the S-S point of view, as a matter of developing cognitions or ac- 
quiring bits of knowledge about the world. This difference has led to 
rather different experimental efforts to substantiate the two views. 

The S-R noncontinuity position maintains that conditioning is com- 
plete in a single trial ; that is, that the connection between a response and 
the stimuli in whose presence it occurs is fully established in just one 
such pairing. It is possible within suek a theory to account for the gradual 
course of most performance functions on the further assumption that the 
stimulus situation varies from trial to trial. 

The essential nature of these theories is as follows : the stimulus side of 
learning is regarded as a large, but finite, collection of stimulus elements 
comprising a stimulus population. On any given trial only a sample of 
the stimuli m the population is assumed to be present. Those which are 
present are conditioned to the response on that trial. After the first trial 
the stimulus sample will consist of some stimuli which have been nre- 
viousiy conditioned to the response and some which have not. With com 
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turned practice the number of stimulus elements not conditioned to the 
response will dimmish, and the number winch have been conditioned will 
increase, until the entire population of stimuli has been conditioned to 
the response Assuming that the proportion of previously conditioned 
stimuli present m the sample on any tr ia l directly represents the prob 
ability that a conditioned response will occur on that trial leads to the 
prediction of a gradual increase in response probability with practice 
The exact form of the function will dep en d upon the trial to trial van 
ability in the stimulus situation If the situation is perfectly controlled 
so that the sample of stimuli is exactly the same from trial to trial, one 
trial learning should occur In a general way, this prediction is that learn 
mg will be more rapid in a very constant situation than it will be in a 
highly variable one There is considerable reason to accept this idea as a 
valid one "We have previously seen that extraneous stimuli disturb the 
course of acquisition through external inhibition In order to guard 
against the effects of uncontrolled disturbances of this sort, Pavlov 
eventually had to build a laboratory carefully shielded from outside 
stimulation 

Voeks (1954, 1955) has made a direct test of the effect of variability 
in the stimulus situation upon eyelid conditioning attempting to control 
internal, proprioceptive stimuli as well as external stimuli To provide a 
condition of maximum control, she had the subjects (1) close their eyes 
between trials, opening them at a signal as the trial began, (2) inhale 
and hold their breath during the trial and (3) press down on two keys 
which turned on the CS, a buzz The UC§, an airpuff, was debt cred 45 
sec later Under these conditions about half of the subjects showed step 
wise acquisition functions which rose from 0 to 100 per cent on a single 
trial Voeks interprets these results as representing an instance of one 
trial learning Before such an explanation can be accepted, however, 
several alternatives will have to be ruled out One of these is that the 
subjects may have responded v oluntanly Another is that the responses 
counted as conditioned responses were unconditioned responses related 
to the key press This possibility was developed earlier (p 59) in con 
nection with the problems of specifying the conditioned response Still 
another explanation has been offered by Spence (195G, pp 108 109) and 
will be presented here because of its potential applicability to a variety 
of problems 

This explanation makes use of a version of Hull’s (1943) continuity 
theory of learning Hull assumed that learning is a gradual process in 
which habit strength accumulates as a simple positive growth function of 
the number of reinforced trials Before habit reveals itvlf in perform- 
ance, however, it is transformed into reaction potential through its inter- 
action with such variables as generalization and motivation Then m 
order to predict whether or not a response Will occur on a given trial (to 
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obtain a probability of occurrence measure), it is necessary to consider 
the effects of two additional intervening variables, threshold ( S L R ) and 
oscillation ( s O a ). Very briefly, a response occurs if reaction potential is 
suprathreshold, but on a given trial whether it is above or below threshold 
depends in part upon the oscillation function. 

The essential elements of this position appear in Figure 5.12. Reaction 
potential, it will be seen, increases steadily with practice, rising above 
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not) to the further assumption that tile breadth of the oscillation func- 
tion varies directly with the number of uncontrolled conditions of the 
experiment In terms of Figure 5 12, this is to say that the area of the 
bell shaped curves would vary inversely with the amount of control m 
the conditioning situation With perfect control the oscillation function 
would disappear completely, with less and less trial to trial stability, its 
area would progressively increase 

Such reasoning makes it possible to explain stepwise conditioning 
curves within the framework of a continuity theory The case is clearest 
for the situation of perfect control Under such circumstances, condition 
mg would reach 1QQ per cent on the trial on which the reaction potential 
function first rose above the reaction threshold, as on Trial 3 in the condi 
turning situation depicted schematically in Figure 5 12 Such control, of 
course, cannot easily be obtained in conditioning experiments Even those 
m which the greatest pains have been taken to insure stimulational sta- 
bility can do no more than reduce the variations in the situation from 
trial to trial to a practical minimum Because of this, not all subjects 
■would be expected to show step wise acquisition and there would be occa 
sional failuies following the first successful response It thus develops 
that extensions of Hull’s and Guthrie’s theories make similar assumption s 
of the operation of a random process and that sudden learning is pre 
dictable from either type of theory It is perhaps important to note that, 
in the accomplishment of this goal, the two theories become surprisingly 
similar, considering the fact that they began with opposite initial 
assumptions 

The other, S S, noncontinuity view represents an extension of Lashley's 
(1929) opinion that “[The form of the learning curve ] suggests that the 
actual association is formed very quickly and that both the practice pre 
ceding and the errors following are irrelevant to the actual formation of 
the association ” Its more familiar form was developed by Krechevskj 
(1932a, 1932b) According to tins theory learning consists in the acqmsi 
tion of knowledge about the learning situation, or of expectancies dev el 
oped through experience with the environment The process is one in 
which the learner theoretically tests and rejects a series of hypotheses 
until the correct one is hit upon “Learning consists of changing from 
one systematic, generalized, purposive wa> of behaving to another and 
another until the problem is solved ” (Krechevskj, 1932a p 332) Obu 
ously such a description of learning is more applicable to complex situ i 
tions where alternative wajs of responding are possible than it is to the 
conditioning situation where response possibilities arc limited The dis 
crimination learning experiment provides the situation in which the 
majority of tests of the cognitive view have been made 
Krechevsky’s analysis of discrimination learning breaks the process 
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down into two stages or periods, a p resolution period, in which erroneous 
hypotheses are tried out, and a solution period, in which the subject 
adopts, and pei’haps strengthens, the correct hypothesis : 

Once the animal is immersed in a given problem-situation, the animal selects out 
of the welter of possible stimuli certain sets of diseriminanda to which he reacts. 
Each time (while “paying attention to” this particular set of diseriminanda) he 
makes what proves to be a “correct response,” he learns (wrongly perhaps) 
something about the significance of this particular stimulus each time he makes 
a “wrong” response he learns something else, but he does not learn anything about 
the “ correctness ” or “wrongness” of the to-be-finally-learned set of diseriminanda. 
Eventually he gives up responding to his first set of diseriminanda and responds 
to another set, and another set, etc. (Krechevsky, 1938, p. Ill) 


Two features of this description are worthy of special note. The first 
is that Krechevsky regards the behavior of the subject during the pre- 
solution period as systematic, although wrong. There is considerable 
evidence that this opinion is a sound one, in that animals show consistent 
position preferences, brightness preferences, or tendencies to alternate 
during the early stages of discrimination learning. Krechevsky used such 
evidence to provide an operational definition of hypotheses as any re- 
sponse tendency which appeared with a greater than chance expectancy 
(Krechevsky, 1932a, 1932b). The other point to note is that incorrect 
hypotheses (responses while the animal is paying attention to the in- 
correct stimuli) have no effect upon the later mastery of the correct 
( finally -to-be-leamed) response. That is, they are “irrelevant to the 
actual formation of the association.” It is this feature of Krechevsky ’s 
position which makes it a noncontinuity one. A continuity view employ- 
ing the concept of hypothesis is entirely feasible although no one seems 
to have espoused one. 


The continuity theory (Spence, 1936) differs from that of Krechevsky, 
specifically in regard to what happens during the presolution period. It 
holds that while the animal is responding incorrectly, but systematically, 
the tendency to respond to the reinforced stimulus is increasing and other 
tendencies are being extinguished. To illustrate, suppose that the learn- 
ing situation requires a black-white discrimination and that black is posi- 
tive. Suppose further, that a certain subject has a strong left position 
preference and responds to the stimulus on the left on every trial for the 
first 40. Since the black and white stimuli are randomly shifted from side 
to side the subject will be reinforced in the presence of the black 20 times 
and according to the continuity position, there will develop an increas- 
ing y strong tendency for the subject to respond to the black stimulus 
Eventually this tendency will be greater than the left-gohm tendency 
and will appear, perhaps suddenly. This differs from the view of Kre 
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hevsky, ivl,o would Hold that the black stimulus would acquire no 
trength until the animal began to pay attention to it 
This specific difference has led to a number of studies which hare 
mployed a discrimination reversal procedure to determine whether the 
•emforced stimulus has acquired a degree of strength during the pre- 
.olution period Suppose, in the hypothetical example above, that the 
reinforcement contingency were switched and the animal now rewarded 
ror responding to the white stimulus rather than the black one Krechev- 
jky ; s noncontinuity position leads to the prediction that this alteration 
would have no effect upon the development of the discrimination Con- 
tinuity theory, on the other hand, leads to the prediction that this re 
versal would interfere with discrimination learning through the mech- 
anism of negative transfer 

Many experiments of the general design just described have been per- 
formed The first was that of McCulloch and Pratt (1934) which 
obtained results consistent with Spence’s position Krechevsky (1938) 
performed two experiments, one supported his view, the other did not 
Subsequent investigations by various authors have most frequently sup 
ported the continuity position 4 

Recently a new dimension of complexity has been added to the con- 
tinuity noneontmuity issue as a result of the work of Harlow (pp 385 
391) and others on learning sets or learning to learn Specifically, it has 
been demonstrated that animals given 200 to 300 discrimination prob- 
lems to solve, over a long series of sessions, gradually acquire the ability 
to solve them suddenly Somewhat analogous results have been obtained 
for other species, using other procedures Hufort, Guttman, and Kimble 
(1954) ran an experiment in which rats were required to learn a position 
discrimination, for example, to choose the right hand goal in a discnmi 
nation apparatus When this had been mastered, reinforcement was 
switched to the other goal box and the subjects learned to choose this 
new goal box "When the subjects had learned this, the problem was 
reversed again and so on through a senes of 10 discrimination reversals. 

On the last three discriminations, the rats all responded correctly after 
the first nonreinforced choice of the previously correct goal box Results 
of tins same sort have also been reported by Buytendijk (1930), Krechev- 
sky (1932c), and North (1950a, 1950b) 

Studies of repeated conditioning and extinction also provide evidence 
of an improved ability to learn as a result of experience in the learning 
situation Following extinction, reconditioning is very rapid, even though 
time for spontaneous recovery has not intervened As specimen findings 
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involving just one reconditioning series, the results of two experiments 
with dogs may be cited. Eineh and Culler (1935) report for flexion 
responses a reconditioning after extinction in from 20 to 25 per cent o 
the original trials, whereas Hilgard and D. G. Marquis (1935) report 
for eyelid responses relearning in from 20 to 40 per cent of the original 
trials. This effect becomes more marked when additional series of extinc- 
tions and conditionings are used. Brogden, Lipman, and Culler (1938) 
report that one dog required the following numbers of reconditioning 
trials to reinstate the CR: After a first extinction, 18 reinforcements; 
after a second, 11; after a third, 5 ; after a fifth, 1 ; after a seventh, 2. 
Bullock and Smith (1953) performed a similar instrumental condition- 
ing experiment in which they conditioned and extinguished rats in the 
Skinner situation. Rates of responding for 10 successive conditioning 
sessions were 1.0, 4.6, 6.5, 7.3, 7.8, 7.2, 7.5, 7.1, and 7.8 responses per 
minute. All of these data thus show that practice at learning a given 
kind of task increases the probability that it will be learned suddenly. 
In summary, this suggests that the naive subject learns gradually and 
that sudden learning, when it occurs, is the result of learning to learn. 


SUMMARY 

The general question with which this chapter has been concerned 
involves the form of the learning curve. Prom the beginning of the study 
of learning this question has been a matter of special interest because it 
touches one of the basic properties of the learning process. For much the 
same reason it has been a matter of theoretical controversy. In spite of 
a great deal of experimentation and debate, however, there is as yet no 
agreement on an answer. 


The Problem of Measures 

It will be possible to determine the form of the acquisition function 
(if there is just one) only when we have available satisfactory measures. 
The need is for indices which are (1) psychometrically adequate and 
which also (2) accurately reflect the underlying process. The question 
of psychometric adequacy leads to investigations of the reliability of 
the various measures of learning. Such studies indicate that these meas- 
ures can attain satisfactory degrees of consistency. Similarly, judiciously 
selected measures are capable of reflecting differences in the amount of 
learning at all levels. 

The second consideration, which requires our measures to reflect tie 
process of learning accurately, poses questions whieh are harder to 
answer. Smee the available measures are always measures of perf"mance 
these questions all involve the problems of transforming such measures 
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into units appropriate to describing the degree of learning At the 
present time such units have not been developed In fact there is not 
even complete agreement as to exactly what constitutes a measurable 
response in the learning situation For example given the fact that with 
practice an organism can learn to respond rapidly vigorously, or per 
sistently, is it appropriate to use speed vigor and persistence as if they 
were quantitative reflections of the strength of some underlying process? 
Or is it eventually going to be necessary to adopt a more molecular 
approach in which responses with different quantitative characteristics 
will be treated as different responses ? Similarly, since the various avail 
able measures do not respond in the same ways to manipulations of such 
variables as drive strength and numbers of reinforcements is it proper 
to consider all measures as revealing the degree of learning m the same 
way? Along the same lines correlational studies have found so little 
correspondence among response measures that it seems improbable that 
they can be regarded as all reflecting the same thing When one asks 
which of the possible measures is best it turns out that the answer is 
different for different situations, and that it depends in part upon prac 
tical considerations 

Learning vs Performance 

A large part of the difficulty of studying the course of learning stems, 
as we have seen from the fact that leammg and performance are difficult 
to separate Attempts to untangle their individual contributions have 
often taken the form of factoriaily designed experiments In these expen 
ments different groups of subjects are first trained under different ex 
penmental conditions. Later on these main groups are divided into sub 
groups some continuing under their original conditions others switching 
to the alternative conditions Appropriate statistical treatment makes it 
possible under certain circumstances to determine whether a i unable 
influences learning or merely performance In effect these statistical 
techniques ask whether a variable produces a relatively permanent effect 
upon behavior as the definition of learning requires. If such an influence 
can be detected a particular variable is said to influence learning If it 
cannot the variable is said to affect performance 
Variables which presently seem to be performance variables on the 
basis of such studies, include (I) the intensity of the conditioned stimu 
lus (2) motivation (3) the amount of reinforcement and (4) distnbu 
tion of practice We should recognize hovrever that the factorial method 
is completely satisfactory only under limited conditions where the out 
come of the experiment provides a clear indication that the variable is 
a performance variable Moreover, the concept that these variables have 
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their effect exclusively upon performance is subject to revision on the 
basis of later experimental evidence. 

Continuity vs. Noncontinuity Theory 

There are two general theories of the form of the true learning 
unction. One of these (continuity theory) takes the typical learning 
curve at face value and holds that learning is a gradual process. The 
other (noneontinuity theory) maintains that learning is sudden and 
insightful. Noneontinuity theories of two rather different sorts have 
been developed within the more general framework of S-R and S-S 
learning theories. The S-R noneontinuity position accounts for the 
gradual nature of most acquisition functions by the assumption that the 
stimulus situation changes from trial to trial, together with the further 
assumption that the strength of a learned response depends upon the 
proportion of these stimuli which have previously been conditioned to 
the response in question. Quantitative statements of this theory turn out 
to have much in common with certain versions of S-R continuity theory. 

The S-S noneontinuity position explains the gradual form of the 
typical learning curve on the assumption that learning requires the 
testing and discarding of successive hypotheses until the subject hits 
upon the correct one. Although there is no doubt that hypothesis testing, 
as operationally defined in the S-S theory, occurs, tests of deductions 
from this theory for discrimination-reversal experiments have favored 
continuity theory rather than the S-S noneontinuity position. 

Some of the disagreements between the continuity and noneontinuity 
positions may ultimately be resolved on the basis of experiments on 
learning sets. These studies have demonstrated that practice improves 
discrimination learning to the point where even lower animlas can solve 
a discrimination problem in one trial, if they have had considerable 
experience with problems of the same sort. Similar demonstrations have 
been made for other forms of learning as well. This suggests that in- 
sightful learning occurs as a result of abilities developed with practice. 
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The Parameters of 
Reinforcement 


The term reinforcement has two different meanings As with the term 
learning, it is important to distinguish between its theoretical and its 
factual, or empirical, meanings In its factual sense, reinforcement refers 
to any of a wide variety of conditions which may be introduced into the 
learning situation to increase the probability that a given response will 
reappear in the same situation The theoretical meaning of reinforcement 
varies considerably from one systematist to another Hull (1943) equates 
it with drive reduction, Skinner (1938) and Tolman ( 1932 ) deal with 
it in stimulus terms, for Thorndike (1911) it is a satisfier or annoyer, 
Guthrie ( 1935 ) regards it as something which alters a situation, Sheffield 
(1948) and Denny and Adelman ( 1955 ) view it as an ehcitor of behavior 
These different views of the nature of reinforcement raise a number of 
basic problems to which we will turn in later chapters In this chapter 
we will discuss a set of studies which require no theoretical commitment, 
being concerned most directly with an empirical description of the 
operation of reinforcement Such researches have had as their aim the 
delineation of a set of functions describing the manner in which certain 
parameters of reinforcement control performance in the learning situ* 
ation The four main clusters of such parametric studies involve (1) 
amount (quantity and quality) of reinforcement, (2) delay of reinforce- 
ment, (3) the mterstimulus interval in classical conditioning, and (4) 
schedules of reinforcement We will deal with each of them in some detail 


AMOUNT OF REINFORCEMENT 

The phrase amount of reinforcement covers two rather different prop- 
erties of remforcers, which have sometimes been referred to as the quan- 
tity and quality of reinforcement The quantity of reinforcement is easily 
specifiable in terms of some physical measure such as weight or volume 
Quality of reinforcement, on the other hand, is usually specified in terms 
of the organism’s preference A reinforcer high in qualit} is one which 
137 
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the organism chooses in preference to others. P. T. Young (1945, 1947) 
has carried out a very extensive series of researches investigating the 
food preferences on one species, the rat. 

In the previous chapter several of the studies of the effects of the 
amount of reinforcement were reviewed, chiefly for the purpose of setting 
forth the view that this variable seems to function as a performance 
variable. We are concerned here with the nature of the function which 
relates performance to the amount of reinforcement. In general these 
studies have yielded results in agreement on the point that performance 
increases as a negatively accelerated function with increases in the 
amount of reinforcement. Among the investigations leadin- to this 
conclusion are those of Gantt, described by Hull (1943), Crespi (1942), 
Hutt (19o4) and Zeaman (1949a) for quantity of reinforcement and 

Hutt’s T I 3) Hutt (1954) for of reinforcement. 1 

Hutt s investigation is of special interest because it is one in which 

amount of reinforcement was varied in two ways, by manipulating 
quantity and quality of reinforcement in a factorially designed experf 
ent. Hutt used a semiliquid mixture of milk, flour, amf water as a 
basic reinforeer. To produce three variations in aua itv nf T u 
mixture, Hutt added citric acid to it to make it sour ! i f ^ baS1C 
for one group of rats; for another 
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TABLE 12 

Mean Rates of Bar-Pressing Under Different Combinations 
of Amount and Quality of Reinforcement (ffutt, 1954) 


Quantity 

Citric 

Sasic 

Saccharin 

If can 

Small 

20 

30 

50 

39 

Medium 

42 

69 

80 

64 

Largo 

82 

111 

13 8 

110 

Mean 

51 

73 

89 



difficult to Specify and control the intensity of most negative remforeers 
(2) There is a natural reluctance on the part of experimenters to employ 
strong negative reiuforcers (3) In an important theoretical way, the 
psychological significance of variations in the strength of a negative 
remforcer is ambiguous This is because the noxious stimuli employed 
m escape, defense, and avoidance learning seem to serve both as motiva- 
tors and as reinforcers 

Such evidence as is available on the effects of the amount of negative 
reinforcement is quite consistent with data from studies employing posi- 
tive reinforcement Increasing intensities yield increasing amounts of 
conditioning (Boren, Sidmaii, and Herrosfcem, 1959, Passey, 1948, 
Prokas y, Grant, and Myers, 1958) , and there is some evidence that the 
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typical function is asymptotic beyond a certain intensity value. Kimble 
(1955) conditioned rats to rotate a wheel to avoid shock, using different 
shock intensities as the unconditioned stimuli for different groups of 
rats. The effect of these different shock intensities upon reaction latency 

appear in Figure 6.1. 


DELAY OF REINFORCEMENT 

Several independent lines of evidence suggest that responses spatially 
or temporally near reinforcement are learned more quickly than re- 
sponses remote from reinforcement: (1) Tolman (1934b) showed that 
rats, running a maze which required two black-white discriminations, 
one 5 after the other, learned the second of two discriminations more 
easily than the first. (2) It is well known that blinds in a complex maze 
tend to be mastered in a backward order (Montpellier, 1933). (3) Rats 
tend to take a short path to a goal rather than a long one (Yoshioka, 
1929; Grice, 1942). (4) If, during the extinction of a running response 
in a straight alley, rats are blocked at various distances from the goal, 
extinction is more rapid if the block is placed near the starting box than 
if it is near to the goal box. (Lambert and Solomon, 1952). (5) There 
is a clear-cut speed of locomotion gradient in the behavior of a rat in a 
straight runway. The animal runs faster as it nears the goal (Hull, 
1934c). All of these suggest that learning (or, perhaps, performance) 
varies directly with the immediacy with which reward follows the 
response to be learned. This goal gradient, or delay-of -reinforcement 
gradient, represents one of the most important applications of condition- 
ing principles to more complex behavioral situations. The goal-gradient 
hypothesis as Hull originally formulated it is as follows: 

The mechanism . . . depended upon as an explanatory and integrating principle is 
that the goal reaction gets conditioned the most strongly to the stimuli preceding 
it, and the other reactions of the behavior sequence get conditioned to their stimuli 
progressively weaker as they are more remote (in time or space) from the goal 
reaction. (Hull, 1932, pp. 25-26) 


Two ideas are important in this statement. The first is the delay-of- 
reinforcement principle itself. The other is the implied analysis of a 
serial chain of behavior into smaller stimulus-response elements This 
latter conception is basic to many of Hull’s earlier theoretical treatments 
and represents an interest to which he returned later (Hull, 1952). 

In many experiments the learned activity is more than a single re- 
sponse such as an eyelid closure or a finger withdrawal. This is particu- 
larly true of instrumental conditioning, in which the response involves 
a series of movements. The rat in the Skinner apparatus secures food by 
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talang successive steps to approach the region of the lever liftin* its 
forelegs from the floor, pressing downward on the lever, lowering its 
body to the food cup, lowering its head, and opening its mouth to seize 
the food In a straight alley, the locomotion of the rat from one end to 
the other may be considered a series of responses by analyzing the speed 
of running separately for the successive segments of the total path 
When an instrumental act is considered as a single response, it is 
possible to apply the principles of conditioning directly For example, 
the maze habit, considered as a unit, will be acquired more rapidly if the 
reward is large and the animal is hungry Gradual disappearance of 
the reaction will occur with trials on which the reward is omitted and 
spontaneous recovery from this extinction will tale place after an inter- 
val of no training There are many facts about maze learning, however, 
which cannot be encompassed by this direct application of conditionmg 
principles to ^he reaction as a unit There is no possibility of explaining 
why the rat takes a shorter rather than a longer route through the maze, 
why some blind alleys are eliminated sooner than others, why mazes of 
different pattern vary in difficulty The answers to such questions can 
be obtained only if the total maze reaction is analyzed in terms of its 
component parts and the stimuli to which these part reactions occur 


External stimuli 


The organism 

FIGURE 6 2 Stimuli and responses involved in a serial act The reactions R lt 
R-, R 3 , R 4i K, to the external stimuli Sj, S 2 , S 3 S 4 , S 5 become integrated as 
a senes by the proprioceptive stimuli 8 i, 8 2 , 63 , s., from the muscles used in 
the reactions Original responses are represented by solid arrows, learned re 
sponses by broken ones According to the goal gradient hypothesis the 
strength ol the connection S 5 R B >S 4 R 4 >S 3 R 3 , and so on and similarly 
s 4 R 5 >s 3 R 4 » and so on Alter Hull (1930a) 

Sueh an analysis, that of Hull (1930), in conditioning terms appears 
in Figure 6 2 Si, S 2 , and so on represent the successive external stimuli 
provided by the environment, each evoking in the organism a separate 
response, Rj, R 2 and so on Each of these responses produces a elnrac 
tenstic proprioceptive stimulus such as s„ s 2 Thus, in the chain of 
responses, 1 given reaction occurs m fairly close temporal proximity to 
a complex of stimuli which is partly external and partly internal Learn- 
ing consists in a strengthening of the tcmlencv for these environmental 
and proprioceptive stimuli to evoke the appropriate response The arrows 
m Figure 6 2 represent the stimulus response connections in question 
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Restated, the goal-gradient hypothesis is that these connections are more 
readily established for stimuli and responses near the goal than they 
are for stimuli and responses early in the sequence. 


Learning the Shorter Path to a Goal 

A common conception of maze learning is that it is a complex case 
o earning of the shortest of several possible routes to reward. Errors 
such as entries into blind alleys, retraeings, are actually reinforced, but 
a correct run results m reward sooner. According to the goal-gradient 
principle, the responses involved in a correct run should, therefore be 

“w; ( rf Fi ""I 6 f - M ° re animaIs ^uld learn to 

take the shortest of several alternative paths to a <*oal 

Yoshioka (1929) performed a test of this deduction. Rats in his experi- 
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meat ran a maze in which there were two alternative paths to food The 
longer path was 211 inches from start to goal, for another maze of the 
same pattern, the longer path was twice as long 422 mches In either 
maze, the other path could be shortened to any desired length YoshioLa 
tested rats m these two mazes, using ratios of long to short paths of 1 07, 
1 14, 1 23, 1 33, and 1 44 He found (1) that rats did learn to take the 
short path, (2) that the rats took more trials to learn the short path 
when the ratios were small than when they were large, and (3) that 
equal ratios were equally difficult to learn in the short and m the long 
mazes This led him to propose that it was the relative response strength, 
produced by differing delays of reinforcement, that determined the 
ease or difficulty of acquiring the short path habit, and that the ease of 
learning to take the short path is independent of the lengths of the paths 
According to this hypothesis learning to take the short path would he 
just as easy in a very small maze as m a very large one as long as the 
ratios of short to long paths were equal More recent evidence (Anderson, 
1933, Grice, 1942) suggests that this specific interpretation is wrong 
and that the ratio and the absolute length of paths combine to determine 
the number of trials required to learn to take the shorter path 


The Form of the Delay-of-Reinforcement Gradient 
Such evidence, together with the results of temporal delay of reinforce 
ment studies to be considered presently, led Hull (1943) to propose the 
following equation as relating the limit of habit growth (AT) to delay 
of reinforcement 


M' — MX 1 0~ RT 

where M is the maximum habit strength possible (in arbitrary units 
called habs), given the other conditions of the experiment (amount of 
reinforcement, and so on) T is delay of reinforcement, and Ji is an 
empirically determined constant The form of this equation is that sliouu 
in Figure 6 4 for the case where Af = 80 and K — 00G72 

One application of this function explains the tendency of an animal 
to take the shorter of two alternative paths to a goal in a maze To dlus 
trate this application, let us consider a hypothetical experiment in which 
five different gioups of subjects learn to take the short path in fne 
different mazes which differ in absolute size while the ratio of delays of 
reinforcement remains constant at 2 to 1 To be specific let us suppose 
that choosing the long path means that reinforcement is dehied by 
10, 40, 160, 240, and 360 seconds Choosing the short path h ids to 
delays of 5, 20, 80, 120, and ISO seconds, and the ratios of delavs in the 
five mazes are 10/5, 40/20, 160/SO, 240/120, and 060/180 A glance at 
Figure C 4 will show that the animals should eventual!} conie to proftr 



144 


CONDITIONING AND LEARNING 



FI Swm^ T V & eXp ° nential , delay-of-reinforcement gradient proposed by 
Hull (1943 For purposes of dealing with problems in the text, values of 
maximal habit strength may be estimated by reading from a given delav value 
to the curve and from the curve to the vertical axis of the graph 7 


the shorter path in each of these eases because the limit of habit growth 
18 ^-ter with the shorter delay of reinforcement. The more s^ecTfic 
question of the degree of this preference depends upon the manner in 
wnch the conflict between the tendency to take the short path and that 
to take the long path is resolved. In order to maintain consistency with 
other aspects of his theory, Hull assumed that if +i 7 
the two habit strength, not th 
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are also in line with this interpretation 6 ° sllloka ’ s original data 
were within the fairly wide ran^e of \ + assurain S that his maze lengths 
ratios are nearly equally easy to discHmi^te^^ dlStaUCes where e( l ual 
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Tlie Order of Elimination of Blinds 

A similar application of the goal gradient principle predicts the back 
ward order of elimination of blinds in maze learning To illustrate, sup 
pose that entering a blind alley adds 5 seconds to the time required to 
run the maze, and therefore, 5 seconds of delay of reinforcement In short 
a correct response is always rewarded 5 seconds sooner than an error 
Now let us consider the situation as it theoretically exists at choice points 
three different distances from the goal a near, intermediate, and remote 
choice point Por illustrative purposes, assume that the correct response 
at the choice point near to the goal is rewarded after only 2 seconds, an 
error will be reinforced, therefore, after 7 seconds Assume further that 
the intermediate and far choice points lead to reinforcement after 80 
and 120 seconds for a correct response This means that errors are rein 
forced after 85 and 125 seconds respectively The delays of reinforcement 
associated with correct and incorrect choices at these three points in the 
maze are, thus, 2 and 7 seconds, 80 and 85 seconds, and 120 and 125 
seconds Reference to Figure 6 4 will show quite clearly that the differ 
ence in habit strength is greater in the case of the two shortest delays 
and becomes less and less for the pairs of longer delays Therefore, the 
choice point near to the goal should be easiest to learn The intermediate 
and far choice points should be progressively more difficult, because the 
maximal values of habit strength are more nearly the same 
Experimental tests of this deduction from the goal gradient principle 
m complex mazes have yielded somewhat ambiguous results and have 
shown that many factors other than this principle enter to complicate 
the picture For example, certain purely physical factors are important 
in controlling choice point behavior These include inertia which favors 
a forward going tendency and centrifugal swing (Schneirla 1929, Bal 
lachey and Buel, 1934) which leads the animal to run off at a tangent 
after a turn and to take the opposite turn at the next choice point 
Beyond this animals tend to orient m the direction of the goal box, or 
sometimes their home cages, leading to errors into blinds pointing in 
that direction Finally and most important, they tend to make antici 
patory responses, that is, to make the response which is correct at the 
last choice point too soon Because of such complexities it is seldom that 
the blinds in a maze arc eliminated in a perfect backward order, although 
a tendency toward such an order is usually obtained 
Detailed studies of the patterns of errors made by rats learning to 
traverse a maze have attempted to eliminate as manj of these comph 
eating factors as possible But even under the best conditions, the func- 
tions obtained have been complicated Depending upon the particular 
maze pattern, there are too man* or too few errors in the middle of the 
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maze. To account for such, effects it has been necessary to assume the 
operation of additional theoretical mechanisms. The most thorough at- 
tempt to treat the order of elimination of maze blinds is that of Hull 
(1952) who combines the goal-gradient concept with that of the fractional 
anticipatory goal response (i'g) in order to make a series of deductions 
about the ease of learning different portions of the maze. From the goal- 
gradient principle Hull deduces, as we did above, that the response at 
the final choice point in a maze will be easiest to learn. Once it is learned, 
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liowever, it will tend, on the basis of stimulus generalization, to appear 
at other choice points, that is, in anticipation of the final choice point 
Whether such anticipatory response tendencies aid or interfere with 
learning the maze depends upon the maze pattern 
Figure 6 5 illustrates a maze used by Hill (1939), Hull (1947a), and 
Sprow (1947) 2 to study the order of elimination of errors m a linear 
maze As the two schematic ground plans illustrate, the correct path 
through the maze may require either a senes of identical responses 
(homogeneous compound trial andrerror learning ) or a senes of different 
responses (heterogeneous compound trial and error learning) In the 
homogeneous case, anticipatory responses are always correct and aid 
learning, in the heterogeneous case these responses are always antici 
patoiy errors and interfere Figure 6 6 illustrates this point by means of 



FIGURE 6 6 Comparison of the pattern of errors obtained in heterogeneous 
and homogeneous compound trial and-error learning Data from Hull > 

and Sprow (1947) 


the data obtained by Hull (1347) and Sprow (1347) for the hcterogeoe- 

ous and homogeneous cases respectively , 

Tlie general shape of the functions plotted in Figure C C sbous that 
predictions from the goal gradient will be most in error in the nmld «• 
portion of the mare It prediets too many errors in tins region for tl 
homogeneous ease and too few for the heterogeneous one “” u 'j"'' 
theorizes that this reflects the fact prcuouslj mentioned, that aiiticu a 
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tory responses hinder learning in one case and benefit it in the other. 
The problem is to account for the fact that these aids and interferences 
are concentrated in the middle of the maze. Hull shows that the principles 
of st im ulus generalization and stimulus intensity dynamism can be made 
to deal with this fact. In this derivation Hull stresses the control of the 
response by proprioceptive rather than exteroceptive stimuli, because 
the latter are nearly identical at all choice points and, by generalization, 
should tend to elicit the goal reaction equally. The proprioceptive stimu- 
lus situation, however, changes from choice point to choice point in 
important ways. Specifically Hull proposes that the intensity of the 
stimulus traces left by the responses involved in making previous choices 
will be greater in the middle of the maze than anywhere else. This is 
because the first choices occur in the absence of such traces and, by the 
time the last choice point is reached, the strength of the proprioceptive 
traces has already begun to deteriorate. It is for this reason that the 
final correct response shows the greatest tendency to be elicited anticipa- 
torily in the middle of the maze, leading to an excess of errors in the 
case of heterogeneous compound trial-and-error learning and to a pre- 
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poiiderance of correct responses in the homogeneous case A basic assump 
tion in this deduction is that the probability of occurrence of a propno 
ceptively controlled response depends upon the intensity of the propno 
cepiive stimulus trace As we have seen there is reason to accept such an 
assumption for exteroceptive stimuli (p 120) 

There is evidence from brain ablation studies that anticipatory errors 
are to some extent, prevented in normal animals by functions of the 
frontal cerebral cortex Carpenter (1952) trained rats in a four choice 
linear maze in which the pattern of correct choices was right, right, 
right left The purpose of this order was to provide an optimal oppor 
tnmty to study the appearance of the final left response at the other, 
preceding, choice points Some of the animals m this study were normal 
and some had had extensive portions of the frontal or parietal cortex 
removed Figure 6 7 is a presentation of the pattern of errors for these 
three groups The brain damaged animals made many more errors than 
the normal ones but this difference was not statistically significant on 
comparisons of the behavior of normal and parietal animals The annuals 
with frontal ablations were as the graph suggests inferior to either of 
these groups Similar evidence that the frontal cortex inhibits the early 
appearance of the final correct response has been obtained by Crawford 
Fulton Jacobsen and Wolfe (1948) Harlow Davis Settlage, and Meyer 
(1952), and Jacobsen Wolfe, and Jackson (1935) 

The Speed of Locomotion Gradient 
Exactly the same reasoning used in the prediction of the order of 
elimination of blinds applies to the speed with which an animal traverses 
a maze The goal gradient alone leads us to expect ever increasing speeds 
as the animal approaches the goal Considerations involving an apphea 
tion of the principle of stimulus generalization however suggest that 
ESW&d. may bw greater at aovae prvor to the final maze segment than 
it is in this segment itself Such an argument it will be seen, treats speed 
of running as a chaining of homogeneous responses 
Figure 6 8 presents data obtained by Hull on the time spent b} rats 
m various segments of a 40 foot straight runway As anticipated the 
animals spent most time in the maze sections nearest the starting box 
and showed a progressive increase m speed as the} approached the goal, 
slowing up slightly in the final segment All of these features tended to 
bo more marl ed early in training than late The bottom line in higure 6 S 
shows that protracted training flattened the gradient The effects were 
later confirmed b} Drew (193J) who also showed that the gradient occurs 
in mauj situations and is more pronounced under conditions of wass< 
than under conditions of distributed practice The goal gradient Iijpol e 
sis, particular!} the empirical representation of it in the speed-of oco- 
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Wolfe found that delays as short as 5 seconds interfered with the learn- 
ing of a single T-maze by rats. Longer delays, up to 20 minutes, produced 
progressive decreases in the efficiency with which the task was mastered, 
the greater part of the decrease occurring within the first minute. Later 


investigators have pointed out that even this relatively steep gradient 
is probably too extended. Perkins (1947) calls attention to the fact that, 
in such studies, any differences between the delay chambers preceding 
reward and those preceding nonreward can provide the basis for different 
tial secondary reinforcement. In what amounts to a repetition of the 
Wolfe study, with this source of secondary reinforcement eliminated by 
interchanging delay chambers from trial to trial, he obtained a gradient 
which fell to a lower level with long delays than Wolfe’s gradient had. 
Tins comparison appears in Figure 6.9. There was, however, a significant 
amount of learning with a delay of 2 minutes. Perkins attributes this to 
secondary reinforcement provided by the differential proprioceptive cues 
produced by making the correct or incorrect turn in the T-maze. 
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FIGURE 6 10 Delay of reinforcement gradients obtained by Penn (1943a) in 
the Skinner box and Grice (1948) in discrimination learning Grice’s pro 
cedure effectively eliminated secondary reinforcement with the result that 
there is little learning with delays of 5 seconds or longer 

clearly much steeper than any of the gradients previously obtained, a 
fact attributed to the complete elimination of secondary reinforcement 
Both Grice and Perkins ran additional groups of subjects in their 
experiments for the purpose of attempting to assess the importance of 
secondary reinforcement directly Perkins, for example, ran one group 
of subjects using a 45 second delay and very different delay chambers 
following correct and incorrect turns This group learned significantly 
better than the group in the mam experiment in winch the boxes 11610 
identical Grice used two groups for which he introduced secondary rein 
forcement For one of these groups, the correct and incorrect goal boxes 
were the same color (black or white) as the positive and negative stimuli 
For the other group the responses required in traversing the two routes 
to the goal were very different One route required a sharp turn, the 
other climbing an inclined plane With a 5 second delay of reinforce- 
ment, both of these groups mastered the discrimination with greater case 
than the group without such sources of secondarj reinforcement 

Delay of Reinforcement in Aversive Conditioning 
It is impossible to formulate a plau to stud} delay of reinforcement 
in escape and avoidance learning without making an initial commitment 
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to some particular theory of the nature of reinforcement. If one holds 
that the reinforcing power of an aversive stimulus occurs with the onset 
of this stimulus, then one studies delay of reinforcement by manipulating 
the time between the presentation of a warning signal and the onset of the 
aversive stimulus. Experiments of this sort represent a version of the 
interstimulus-interval experiment to he discussed later. 

If, on the other hand, one accepts a drive-reduction theory of rein- 
forcement and regards escape from a noxious stimulus to be the critical 
reinforcing event, the appropriate experiment to use in the investigation 
of delay of reinforcement is one in which shock termination follows the 
response to be learned by varying periods of time. No experiments using 
exactly this procedure seem to have been done. Various studies manipu- 
lating the duration of a noxious TICS in classical conditioning have been 
done, however, (Mowrer and Solomon, 1954; Wegner and Zeaman, 1958). 
They have uniformly found UCS duration to have no effect upon condi- 
tioning. This seems to mean that the critical reinforcing event in classical 
conditioning occurs with DCS onset, and that the drive-reduction analy- 
sis does not apply. 

Instrumental avoidance conditioning provides a somewhat different 
opportunity to investigate delay of reinforcement by manipulating the 
time at which the warning signal terminates. This procedure again as- 
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sumes a drive redaction interpretation of reinforcement Briefly the 
reasoning is as follows the warning signal, through jts association xntk 
punishment, becomes a noxious stimulus, itself Tims the termination of 
this stimulus should be reinforcing, and postponing the time at which 
it terminates should be the same as delaying any other reinforcement 
Kamm (1957a, 1957b) investigated this possibility m a senes of studies 
with rats In his experiments the CS was a buzzer, and the response 
required of the subjects to avoid a shock was that of running from one 
side of a shuttle box to the other For different groups, the offset of the 
buzzer followed the execution of the correct response by 0 0, 2 5, 5 0, or 
10 0 seconds Learning was much better when the response terminated 
the buzzer immediately than it was under any of the other conditions 
Figure 6 11 presents the results from two of Kamm ’s experiments in 
terms of mean number of avoidance reactions The function is quite 
similar to those which ba\e been obtained by other investigators using 
positive reinforcers 


The Delay-of-Pumsliment Gradient 
Possibly because of the undependable effect of punishment upon be- 
havior, the number of studies concerned with a possible delay of punish 
ment gradient is small The experiments of Bevan and Dukes (1955), 
Warden and Dymond (1931), and Kamm (1959) exhaust the list In 
Kamin’s study rats learned to go from one compartment to the other in 
a shuttlebox m order to avoid shock Then, m extinction, fi\c groups of 
animals were punished with electric shock for making the aioidant re 
sponse and one group was not For the different punishment groups the 
shock was delayed by 0, 10, 20, SO, or 40 seconds after the occurrence of 
the reaction As an over all effect, it was found that punishment markedly 
reduced the number of trials to extinction Whereas the control group 
gave about 45 responses, the experimental group showing the greatest 
resistance to extinction ga\e only about 10 Median number of responses 
in extinction for the 0 , 10 , 20 , 30 , and 40 second delay groups were 
3 0, 5 5, 10 5, 9 5, and 9 5 respectnely It thus appears that there is a 
delay of punishment gradient which has some of the same features as 
the more widely studied delay of reinforcement gradient It would be 
interesting to know whether tins gradient is approximately the same for 
the situation in winch the punished response was established with a 
posit i\e reinforcer 


INTERSTIMULUS INTERVAL 
Iu classical conditioning experiments the tune betwetn conditioned 
and unconditioned stimuli is a variable which parallels delay of n-m 
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foreement in instrumental learning. In this ease, however, the delay- 
period is measured from the onset of the CS rather than from the occur- 
rence of a response. 

Pavlov regarded the simultaneous” conditioned response as basic to 
other types and usually established it before proceeding to develop a 
trace or delayed CR. The ordinary procedure for producing a delayed 
conditioned reflex, for example, was to condition simultaneously and 
then to postpone the unconditioned stimulus for five seconds each day 
until the desired delay was reached. By a “simultaneous” CR Pavlov 
meant one in which the CS-UCS interval was 5 seconds or less. He 
apparently found no important differences in ease of conditioning with 
the CS-UCS intervals between 0 and 5 seconds. More recent American 
evidence, however, indicates that the optimal interval is much shorter 
than 5 seconds, probably being something more like a tenth of that value 
Pavlov s success m obtaining conditioning with relatively Ion- intervals 
raises a question to which we will return in the next chapter” when we 

to p :s “ the possibie rewe 01 


TABLE 13 

Results of Ttp.cal Studies oe the Inteestmulus-Inteeval Function 


Intervals in seconds. 

Negative numbers refer 
to backward condition- 
ing procedures. 

~'.30 Jo 5 l o' 1 ' iV 25 ' BeSt T f ondition iag at .30 
’ ’ ' > • sec. Little or none in back- 

ward procedure. No reli- 
able difference between .30 



Fitzwater and 
Riesman (1952) 


Finger 
withdrawal 


0, that is, CS and 
UCS strictly 
simultaneous 


-5, 0, .5 


Fitzwater and 
Thrush (1956) 


CR (perhaps pseudo-CR) 
appeared early in condi- 
tioning and extinguished 
with further practice. 

Confirmed Cohen (above) 
lor 0-second group. Condi- 
tioning with forward pro- 
cedure; not with baek- 



Rest conditioning at .4 
seconds. 
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TABLE 13 — Continued 


Investigator (s) 

Response 

Intends tn seconds 

Negative numbers refer 
to backward condition 
mg procedures Results 

Kappauf and 
Schlosberg 
(1937) 

Leg flexion & 
respiration 

1/3, 2/3, 1, 2, 4 

Flexion response showed 
little conditioning at any 
interval. 

Respiration conditioned 

best at 2/3 second 

Kimble (1947) 

Eyeblink 

1, 2, 225, 250, 

3, 4 

Conditioning best at 4 
seconds 

Rimble, Mann, and 
Dufort (1955) 

Eyeblink 

5, 1 0, 2 0 

Conditioning best at a 
seconds 

Kiab (1955) 

Wheel turning 
to light to 
avoid shock 
(rats) 

25, 1 0, 5 0 

Resistance to extinction of 
avoidant response greatest 
after 25 sec training in 
tervaL 

McAllister 

(1953a) 

Eyeblink 

1, 25, 45, 70, 2 0 

Conditioning beat and 
equally good at 2o and 43 
sec intervals 

McAllister 

(1953b) 

Eyeblink 

45 shifted to 035 or 
25 

Level of response altered 
appropriately with shift 

Moeller (1954) 

GSR 

25, 45, 1 0, 2 5 and Best conditioning at 45 
backward condi see. confirmed Whito and 

tioning control Schlosberg (below) 


Nagaty (19Sla) Instrumental Backward procedure Backward conditioning pro 
wied turn Shock presented 1 duced response decrement, 
to avoid sec before CS in 

shock Non extinction 

shock con 
trol (rats) 


Nagaty (1951b) 

Bar press to 
obtain food 
(rats) 

Food presentel 2 see 
before CS (pres 
entation of bar) in 
extinction 

Confirmed exj enment with 
shock (Nagaty, IOjIo, 
above) 



— 5, — 25, 15, 1 9, 

Best conditioning at 5 see. 

Kellogg (1947) 

withdrawal 

15 

Confirmed Cohen in that 
level of responding de- 
creased with praetteo in 
backward conditions. 

White and Schlos 
berg (2952) 

GSR 

0, 25, 50, 10 2 0, 

4 0 plus pscudocon 
ditiomng control 

Best conditioning at .50 
sec 


Stuclus o£ the intcrstimulus mtcrml ham most otten Inn cJrrml out 
on the conditioning oC motor reflow, although exception, do occur 
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Table 13 presents a sample of relevant investigations in very brief form, 
and Figure 6.12 presents tlie results of three of them graphically. As 
these materials indicate, the most typical finding has been that the opti- 
mal interval is in the quarter-second range on either side of .5 seconds. 
Exceptions to this rule (Kamin, 1954; F. R. Brush, E. L. Brush, and 
Solomon, 1954; Church, F. R. Brush, and Solomon, 1956) have almost 
always come from studies of avoidance learning where the interstimulus 
interval may play a different role than it does in classical conditioning. 
Moreover, these studies have usually faded to sample intervals shorter 
than about 5 seconds. Where such intervals have been employed (Kish, 

1955), there is some suggestion that the optimal interval is less than 
one second. 
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increase m a negatively accelerated way from zero to their maxima, 
there will obviously be a range of nearly optimal intervals (3) When the 
interstimulus interval has been changed in the course of conditioning, 
the result has been a gradual shift m the level of conditioning to a level 
appropriate to the new interval (4) It is a matter of considerable inter 
est that the lnterstunulus interval function for GSR conditioning seems 
to be the same as that for skeletal responses such as finger retraction and 
the eyeblink This is somewhat surprising because the temporal charac 
tenstics of the two types of response are very different, longer latencies 
and durations being typical of autonomically controlled reactions like 
the GSR Two possibilities have been offered as explanations for this 
similarity of function White and Schlosberg (1952) take such results 
to mean that the associations formed in classical conditioning are of a 
stimulus stimulus sort rather than stimulus response so that the precise 
nature of the response would make no difference Moeller (1954) and 
Smith (1954), on the other hand adopt the position that the “condi 
tioned” GSR is not a direct response to the CS but is, rather, an un 
conditioned response which occurs as a by product of unmeasured skele- 
tal reactions which are the true CR’s in these experiments This latter 
position thus explains the unexpected similarity of the mtmtunulus 
interval functions for GSR and skeletal reflexes by the hypothesis that 
both involve motor responses There is little evidential basis for choosing 
one of these explanations over the other (5) It is to be noted that the 
delay of reinforcement function obtained in the absence of secondary 
reinforcement and the mterstimulns interval function appear to be quite 
similar Both Duryea (1955) and Smith (1951) have noted this fact and 
have performed experiments designed to make a comparison of the two 
functions possible 

The reasoning behind these two studies is most easily presented in 
terms of the concrete learning situation (Figure 6 13) which both of 
them employed This apparatus was a T maze in which the color of one 
section of the stem could be changed from trial to trial in order to 
provide a distinctive cue The procedure involved a form of successive 
discrimination (p 362) in which the subjects were required to learn to 
go one way at the choice point if the distinctive section of the stem had 
been white and the other if it had been black To study delay of reinforce- 
ment the interval between response and reward the subjects were con 
fined for a fixed period of time in the section of the maze following the 
choice point To study a process conceived to be analogous to the inter 
stimulus interval, the subjects were delajed in the section before the 
choice point. In both studios delajs of the latter sort were found to 
interfere with learning The results obtained by Smith also suggested 
that the function relating the amount of such interference to time u 
similar to the delaj of reinforcement gradient obtained by Gnce, reach 
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FIGURE 6.13. T-maze used by Duryea (1955) in bis study to compare the 
effect of delay before and after the choice point in a maze. Symbols are to be 
read as follows: S = starting box; D = door; B, G, and W = black, gray, 
and white, respectively; C = curtain. Delay before tbe choice point was 
manipulated by closing doors Do and D 3 to confine tbe animals. Confinement 
following tbe choice-point was accomplished in tbe same way. Color of one 
section of tbe approach alley and tbe goal boxes could be changed so that 
they matched or did not match. 

ing an asymptote in a matter of 5 or 10 seconds. Duryea’s results also 
suggest that a delay of only a few seconds before tbe choice point pro- 
duces a marked increase in the difficulty of learning. But in this experi- 
ment a similar delay of reinforcement following the choice failed to 
impede the progress of learning. 

SCHEDULES OF REINFORCEMENT 

In experimental practice, reinforcement may occur on every trial or 
after every correct response, or it may occur less often than this. The 
first procedure is that of continuous reinforcement ; the second is partial 
(or intermittent) reinforcement. Partial reinforcement is administered 
according to some schedule in which reinforcement is given either on a 
time-contingent basis or for a certain nronortion of correct response s. 
~Tn“ one of theTarliest experiments on partial reinforcement, Pavlov 
(1927, pp. 384-386) found that conditioned responses developed rapidly 
if reinforcement was given only every second or third trial, but failed 
to develop if reinforcement was given only on every fourth trial. Later 
studies employing rats in the Skinner box and in the T-maze (Bruns- 
wick, 1939), eyelid conditioning (Grant and Sehipper, 1952 ; Humphreys, 
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1939a, Kimble, Mann, and Dufort, 1955) and the GSR (Humphreys, 
1939c) m human subjects, flexion responses in dogs with food and shock 
reinforcement (Brogden, 1939a), and an object displacing response in 
monkey (Michels, 1955) have obtained results in general agreement with 
Pavlov’s finding In summary these investigations have shown that ncr 
formance during conditioning is usually, but not always, somewha t 
poorer under partial reinforcement than under continuous reinforc ement, 
and that a marked reduction in level of conditioning occurs when the~ 
proportion of reinforced trials is reduced to 20 or 25 per cent_ Sei 
eral of these investigations have also demonstrated that r esistance to 
extinction is greater following partial reinforcement than following 1 

c ontinuous reinforcement This fact will be considered m more detail in 
Chapter 10 

The Michels study mentioned above investigated response latencies 
of monkeys under 11 different proportions of reinforcement ranging 
from zero to 100 per cent On each trial the experimenter presented the 
subject with a single object in the Wisconsin General Test Apparatus 
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^/(shown in Figure 12.11, p. 386). A food reward was placed under the 
object in the proportion of trials dictated by the experimental condition 
The response measure obtained was the time between the presentation 
of the object and the monkey’s response. Averages of these measures are 
presented m Figure 6.14, where is it quite apparent t hat an increase i n 
responsektencyoccurs in the neighborhood of 20 per cent reinforcement. 
^oiejOje r_ce£rreiniorcement, inereaangvalpiS 
J^erenceJit^ a matter of some interest that this latter result seems to 
ppeaFFnly m mstrumental conditioning situatio ns. Classically condi- 

' I extensive work on achedules^l^&JSiHriiTEaroE 

Skmner and his associates (Skinner, 1938; Ferster and Skinner, 1957) 
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“ a ™ r “ f pigeons and 
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(previously called aperiod ic ). Inrti (2) . VariaUe Interva l 

after a period of time which varies f S SC . edlde5 Enforcement occurs 
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( 3 ) J^e dB atio. In this r^^gnLjgt^v^. j s 60 seconds. 


comprehensive snmma^np^to £* “ 
or reinforcement in Li,-* ^ 00k P re3 ents a large am™!? ? t0 tbe date3 of their 

probably of greater inter 1 + ^ As a treati se on general ° n scheclules 

Skinner (I9 5 f\ mterest today than when it ^ 1 ?j oblems of learning, it is 

— ssesLsrsj: 



the PARAMETERS OF REINFORCEMENT 163 

to reinforced responses is 15 -1.( 4) FomWcMo. Reinforcement occurs 
afteFa number of responses which changes from one reinforcement to 
another. A 15 :1 variable ratio schedule might be produced by randomly 
reinforcing every 5th, 10th, 15th, 20 th, or 25 th response. 

difffirai Lschedules of reinforcement affect responding in charn e. 
teristic ways. Performance on the fixed -interval s chedule often consists 
a pause followed by a positively accelerating rate of respnnce unt il 
the moment of reinforcement The pause is of greater duration for long 
fixed intervals than for short ones and may disappear with extended 
practice. Variable-interval schedules p roduce a pattern of performance 
characterized by constant rates of re sprmriing_r>vpr Inng .periods of time 
Because of the great stability of behavior produced by the variable- 
interval schedule, animals are often trained on it as a preliminary to 
some other treatment such as a test for generalization (Guttman and 
Kalish, 1956) or the administration of a drug (Sidman, 1955c, 1956a). 

By comparison with the interval schedules, fixed-ratio schedules tend 
to produce very high rates of responding If the ratio is a small one, 
responding begins immediately after reinforcement If the ratio is large, 
however, a pause occurs after reinforcement, and responding begins only 
after a delay which increases iwth-iaereases m the size of the ratio (see 
Figure 6.15). The subiect tends in this schedule to respond either at a 
maximal rate or not at all. The same general statement applies to per- 
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formanee under variable-ratio schedules. The pauses, however, tend to 
occurjit _times other than immediately following reinforcemen t. As with 
the fixed-ratio schedule, low variable-ratio schedule s tend to produce 
high response rates without pauses. 

Perster and Skinner (1957) report the effects of several additional 
schedules such as the tandem schedule, in which a single reinforcement 
i s programmed -by two different schedule s operating in succession : rate- 
remf or cement schedules , i n which high or low . rates of resnoM inrifr 
reinforced; m ultiple schedules, in which a signal marks a change from 
one reinforcement contingency to another; chained schedules . in which 
responses m one. schedule produce the stWus which~mdicates that 
another schedule is now in operation, the new schedule being the one on 
which reinforcement occurs; and concurrent schedules, in whir >h t™ nv 
more schedules function at the occurring 

according to both. These schedules have not been investigated so thor 

oughiy as those described previously. The interested reader is referred 
to the Perster and Skinner book. 


SUMMARY 

meaning of tfeterm/reS/oTOm” nf iUs^MTb? theoretieal 

of empirical relationships between certain aspect J “ 3 Tariety 
the rate of learning tVia nav. + , • as P ec ^ s reinforcement and 
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ment, (2) the delay of reinforcement HHi • ^ amount of remforce- 
(4) schedules of reinforcement Thp’ 1 lnterstimulus interval, and 
about each of these are outlined in summary Ki" *° ^ 

Amount of Reinforcement 

The expression, amount of reinforcement 
reinforcers, their quantity and quality The ^ t0 tW ° aspects of 
is specifiable in terms of physical m 7 ' Th quantlt y of reinforcement 
weight, and so on; the quality of ° f T° lume ’ concentration, 

of the organism’s preferences for m' °. rcement ls specifiable in terms 
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meats leads to the conLTn tit witT * a variety of experi- 
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negatively accelerated function. TUs sStem™ .T?® 88 according to a 
forcers as well as for positive reinforc ™ holds for rein- 

chapter that the influence of the amount T' - We haTe S88n in the last 

performance. We shaU see in the TiTifj ' “'“’“ment is mainly upon 
mechanism of secondary reinforcement P the efleet Solves the 



16o 


THE PARAMETERS OP REINFORCEMENT 

Delay of Reinforcement 

Evidence of many kinds indicates that responses which are followed 
by reward immediately are learned more rapidly than responses for 
which reward is delayed On the basis of such evidence Hull was led in 
1932 to formulate the goal gradient hypothesis in which he proposed 
(a) that complex chains of behavior could be analyzed profitably into 
component elements and (6) that the response elements occurring near 
the goal were learned more easily than those remote from the goaf Hull 
showed that applications of this principle could account for the major 
details of the speed of locomotion gradient which is characteristic of the 
behavior of rats m a straight rumvay and the ability of animals to learn 
to take the shorter of two paths to a goal Some of the finer details of 
these same phenomena however can be understood only if the exact 
form of the delay of reinforcement gradient is taken into account 

For this reason Hull proposed later that the delay of reinforcement 
gradient is a decelerated negative function of time when the lei el of 
learning is expressed in terms of theoretical units of habit strength One 
of the predictions to which this reformulation leads is that the ease with 
which the short path to a goal can be learned (in eases where the ratio 
of the lengths of short to long paths remains constant) will vary with 
the absolute lengths of the paths Specifically such learning will be 
easier when the paths are of intermediate length than when they are 
either very long or very short Incomplete evidence supports this predic 
tion 

Other applications of the revised goal gradient hypothesis require 
additional assumptions For example any -version of this theory applied 
without modifying assumptions predicts a strict backward order of the 
elimination of blinds in a maze Tiiat is the final choices will be learned 
first and the initial choices will be learned last t spcnmeutal data 
support tins deduction only m a very general way The failure of con 
formation is due m part to a set of complications which arc peculiar to 
the complex maze and in part to the fact that it is necessary to take 
additional considerations into account to explain the order of eliraina 
tion of errors A combination of the principles of stimulus generalization 
and delay of reinforcement seems capable of predicting the pattern of 
errors m certain simple linear mazes with four choice points 
Experiments designed to determine the exact form of the delay -of 
reinforcement gradient have led to successive revisions douuwnrd of the 
estimated amount of time by which reinforcement can be delayed if 
learning is to occur At the present time it seems unlikely that learning 
can take place at all with dclavs of more than a few *co »ds This >late- 
ment applies to ne„at»e is well as to positive remforccrs Instances of 
learning with protracted delays of reward are always cases where mime- 
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diate secondary reinforcement occurs. This point will be discussed further 
in the nest chapter. 


The Interstimulus Interval 

The interstimulus interval refers to the time between the onset of a 
conditioned stimulus and the onset of an unconditioned stimulus. Strictly 
speaking, this variable applies only to classical conditioning. Studies in 
winch the interstimulus interval has been manipulated have shown that 

he OsTn temP °« al ° f CS and UCS is about -5 seconds, with 

earn^L TT g ^ ®* her lon £ er or sb <*ter intervals lead to poorer 
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Schedules of Reinforcement 
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Secondary Reinforcement 

The fact that some sources of reinforcement are learned is now 
well-established In general, stimuli regularly associated with primary 
reinforcement take on some of the same characteristics as primary 
reinforcement and are referred to as secondary remforcers Saltzman’s 
(1949) study will serve as a reference experiment in terms of which to 
present the methods employed in the study of secondary reinforcement. 
It illustrates a number of important procedural points and also antici- 
pates many of the factual and theoretical issues with which numerous 
later investigations have dealt. 

Saltzmaii, using the apparatus shown in Figure 7 1, began by training 
rats to run the straight alley to obtain food in a distinctive goal box. 
Two of these groups, a consecutive reinforcement group and a control 
group, were treated in exactly the same way during the training phase of 
the experiment. Over a five day period they received 25 rewarded trials 
in which food was, therefore, paired with the distinctive stimuli in the 
goal box. A third group, an intermittent reinforcement group, also 
received 25 reinforced trials. But, in addition, they received 14 non- 
reinforced trials using the same goal box These non-rcinforccd trials 
were placed haphazardly in the series but were always preceded and 
followed by a reinforced trial. A fourth group, a differential reinforce- 
ment group received the same treatment as the intermittent reinforce- 
ment group except that the goal box was discriminably different for 
rewarded and non- rewarded trials. 

Following this training, all of the rats received 35 trials in the U-mazc 
For all groups except the control group the previously reinforced goal 
box was placed at the end of one arm of the U, and a different goal box 
was placed at the other For the differential reinforcement group, this 
latter box was the box used on the non-rcinforced trials in secondary re- 
inforcement conditioning For the control group, both boxes were identi- 
cal but different from the one used in previous training Thus all groups 
except the control group could receive secondary reinforcement by choos- 
ing one arm of the U. These groups were never fed in the l/-maxc The 
control group was reinforced on one side of the U-mare, primarily, with 
food. 

H57 
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maze runway 

FIGURE 7.1. Typical apparatus employed in several studies of secondary 
reinforcement- Animals are trained in the straight maze first, and then tested 
in the U-maze. The areas designated by S are the starting boxes; those marked 
G are goal boxes. This particular maze -was employed by Saltzman (19491 - 

The results of the experiment were as follows : Out of 15 trials, the rats 
in the control group, which received primary reinforcement, chose the 
side reinforced with food on an average of 10.0 trials. The consecutive, 
intermittent, and differential reinforcement groups chose the side leading 
to the previously reinforced (now empty) goal box on 8.3, 9.0, and 10.7 
trials, respectively. Since there were 15 trials in all, and since the U-maze 
is a two-choice situation, a chance performance would consist of the selec- 
tion of either goal box on 7.5 trials. All of the groups chose the previously 
reinforced box sufficiently often to lead to the conclusion, on statistical 
grounds, that the performance was not chance. This means that the 
animals showed at least some learning. For three of the groups the rein- 
forcement for this learning was secondary rather than primary. 

In addition to providing a reference experiment, Saltzman ’s study 
suggests a number of important points which will come up for later dis- 
cussion: (1) The differential reinforcement group performed better than 
the intermittent reinforcement group, suggesting the importance of dis- 
crimination in the establishment of secondary reinforcement. (2) During 
the 15 test trials the number of correct responses of the continuous rein- 
forcement group increased at first, and then decreased, suggesting that, 
for this group, the secondary reinforcement value of the goal box stimuli 
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underwent extinction (3) It is of some interest that secondary reinforce 
ment seems to be at least as powerful as primary reinforcement when 
secondary reinforcement is established by the differential technique 
One feature of Saltzman’s experiment winch is not typical of secondary 
reinforcement studies in general is the use of the primary reinforcement 
control group , according to some authors (Myers, 1958), it has not been 
used often enough Two more common procedures have been the follow 
mg (1) There are two groups of subjects in the experiment One group, 
the experimental group, undergoes secondary reinforcement training in 
which some neutral stimulus, such as a light, a buzzer, or the stimulation 
provided by a goal box, is paired with a primary reinforcer such as food 
Another group, the control group, does not receive such training Then 
both groups of subjects are tested in some situation in which a given re 
sponse is followed by the stimulus whose secondary reinforcement power 
is being tested If the experimental group shows an increase in the fre 
quency of the response m question, and the other group does not, this is 
taken as evidence that the neutral stimulus has acquired secondary rein 
forcing power for the experimental group (2) Again there are two 
groups of subjects In this design, both groups receive the training de- 
signed to establish a neutral stimulus as a secondary remforcer In the 
test, however, only one of the groups receives the secondary remforcer 
for making the measured response The difference in performance between 
the two groups provides the measure of secondary reinforcement 

A final point to be made is terminological In the description of Salts 
man’s experiment, above, we have employed a distinction which seems 
worth maintaining in the discussion of secondary reinforcement Spe 
cifically the term sccondaiy remforcer will be used when the reference is 
to the stimulus, the term secondary reinforcement will be used when the 
reference is to the effect of this stimulus upon the organism’s behavior 


THE NATURE OF SECONDARY REINFORCEMENT 
A primary remforcer, such as food, has its effect at least m part (pp 
242 to 248), because it reduces a related drive hunger This suggests the 
following question Docs a secondary remforcer based on a pnmarj food 
reward have its effect through a conditioned hunger reduction 1 Expen 
meats designed to test tins hjpothesis (Miles and 'Vic! ens, 19 jJ, Calvin 
BickncU, and Sperling, 1933b) have produced negative results by show 
mg that the presentation of stimuli associated with food and eating does 
not reduce the amount the animal eats It is uctcssurj , therefore, to look 
elsewhere for the answer to the question of the nature of secondary 

reinforcement . , . 

In terms of opcrctu,, it is clear tlmt the procedures nimml to pro- 
Mile a neutral stimulus ivitli seconder, rcnforciuf pouem are lliose ot 
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classical conditioning. A neutral (conditioned) stimulus is paired with a 
primary reinforcer (unconditioned stimulus), and the former stimulus 
acquires certain of the properties of the latter. Figure 7.2 makes this 
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122 to 124) In summary, they have shown that changes in the amount of 
reinforcement produce changes m performance which seem best con 
siaered as the result of motivational (incentive) influences 
There is also evidence, but of a somewhat different sort, that secondary 
remforcers can function as incentives One form of support for this be 
lief comes from the repeatedly demonstrated fact that secondary rein 
forcers are capable of providing the instigation for a response For 
example, Estes (1943) first conditioned the bar pressing response of rats 
m a Skinner box on fixed interval schedule After this response was well 
established he gave the animals 30 trials of " discriminate e training" in 
which a tone was presented for 60 seconds, followed by the delivery of 
food In this part of the experiment the bar was absent Observe that this 
is precisely the procedure used to provide a neutral stimulus (the tone) 
with secondary reinforcing power A test of the effect of this training was 
obtained during extinction For some of the animals (experimental 
group) the tone was presented for two ten minute periods during a one 
hour extinction session, for others (control group) it was not presented 
The effect of the tone during extinction was to produce an elevation in 
rate of responding Other studies have shown that the same effect can be 
obtained if the pairing of tone and food occurs prior to the establishment 
of the conditioned bar press reaction (Estes, 1948) or in another situa 
tion (K 0 Walker 1942) All of these studies show that the training 
ordinarily used to establish secondary reinforcement can provide a slim 
ulus with wbat seem to be incentive properties 
Dmsmoor (1950) made a direct comparison of the incentive and see 
ondary reinforcing properties of stimuli After 10 initial reinforcements 
in the Skinner box, 36 rats received discrimination training m which 
light (for half of the subjects darkness 1 ) was made the occasion for a 
response Such training appears to be csscntwl to the establishment of 
incentives or secondary reinforcement (Ferstcr 19 d 1) Specifically after 
30 seconds without response in darkness the light was turned an and the 
next response, necessarily one which occurred to the light, was reinforced 
with food After 200 reinforcements administered in this way, the rats 
were extinguished under one of three conditions (1) For a control 
group extinction occurred in the dark (2) For a second group, cxtmc 
tion also occurred in the dark, but every response produced a light of 3 
seconds duration This was a secondary reinforcement group (3) For a 
third group extinction occurred in the light but every response turned 
the light off for 3 seconds Tins was a condition iti which the onset of t) e 
light after its 3 second cessation provided the occasion or incentive for a 


* Tor halt of the subject* In the Dnumoor experiment conJilioM 
reinforcement occurred in darkne* Becau,c of the . » ttwltf rc,u, *V* 1 ’ if 
joint under ducuasloo, however, the j roecdurc U deputed for the *ubje<U re 
in the light. 
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response. The results of the experiment appear in Figure 7.3. Observe 
that the curves for the two experimental groups are virtually identical. 
In 3 extinction days, the number of responses produced by the secondary 
reinforcement group and incentive groups -were 234.9 and 235 6 re- 
spectively. The control group emitted only 128.5 responses. Obviously the 
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end of a maze ran) eome to contribute to the instigation of the response 
in a different situation {as in tie starting box of the maze) ? 

Faced with problems of this sort, both Seward (1950, 1951, 1952) and 
Spence (1956) have proposed that incentive motivation develops through 
the conditioning of a goal response, R G; to the stimuli m the reinforce 
men! situation, by what appears to be the same process as was outlined m 
the classical conditioning paradigm for secondary reinforcement in Fig 
ure 7 2 Thus these theories contain the proposition that the operations 
for producing iscentne motivation are the same as those for producing 
secondary reinforcement In order to account for the functioning of 
incentive motivation in places remote from the original conditioning situ 
ation, these theorists call upon the concept of stimulus generalization To 
the extent that the stimuli in the starting situation are similar to those in 
the reinforcement situation, there is a tendency for the goal reaction (R 0 ) 
to be elicited Since the primaiy reinforcer is absent, only a fractional 
component (r G ) of the complete goal response can occur It is tins frac 
tional anticipatory goal reaction which is regarded as having motna 
tional properties and energizing the organism’s behavior Why r G should 
have motivational properties remains a question Seward (13566) pro 
poses that incentives contribute to moti\ ation by adding to the animal’s 
excitement Spence prefers to deal with the concept abstractedly but also 
suggests two possible mechanisms one involves the idea that evocation 
of r 0 places the animal in conflict, the other employs Hull’s (1949) con 
cept of stimulus intensity dynamism (V) and proposes that the stimuli 
s G produced by r G add to the intensity of the stimulus complex control 
ling the animal’s behavior and energize behavior in this way 


Secondary Reinforcement Cased on Negative Primary 
Reinforcement 

The implication in the previous discussion that secondarily rcintorc 
mg stimuli have two functions rewarding and motivating finds a parallel 
in connection with the cues associated with negative reinforcers such as 
electric shock In fact the situation may be even clearer in this context 
because the motivational and rewarding aspects of pain stimuli arc rnoix 
sharply separable than they usually ore in the case of positiv e remforarv 
The motive is aroused by the onset of the noxious stimulus, reward or 
satisfaction accompanies its offset With this distinction made clear it w 
easy to see that (1) stimuli associated with the onset of pain might U 
expected to acquire drive properties, and (d) stimuli associated with Ut 
cessation of pain might be expected to acquire positive reinforung I*wp 
ernes Mowrer (WjG) describes this possibility, referring to the two U«» ** 
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of learning as leading, respectively, to the development of “fear” and 
“hope.” 2 

The bulk of experimentation involving noxious stimuli has been con- 
cerned with the first of these relations. Since it represents a clear case of 
drive conditioning and has always been regarded as such, discussion of 
typical experiments has been postponed to Chapter 13 on motivation. In 
anticipation of more extended treatment, these studies show that neutral 
stimuli paired with painful stimuli do take on drive properties and that 
aeir dmmaton is reinforcing. That is, after such pairing, presenting 
the previously neutral stimulus will lead the organism to make response 

r^mult 8 ‘ “ ftey ^ “ by thc “ «,e 

si»aUta^eTe™ eXP t erime f S - e 1 ng the ° ther that stimuli 

» a mg the termination of pam become positive secondary reinforcers 

With occasional exceptions (Nefzger, 1957), however, such stupes as 
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there are have yielded positive results Mowrer and Aiken (19o4) taught 
rats to press a bar to obtam food m the first stage of an experiment In a 
second stage they paired a three second flickering light with a ten sm and 
shock under the 5 arrangements shorn m Figure 7 * ^ vrhlch CS ( ) 

terminated as the UCS came on, (2) came on with the i UCS, , (3) came 
3 seconds before UCS termination and went off with the UCS, (-4) 

a Ttno *■ rsf? and (*>1 came on 2 minutes after the offset of 

procedures was tested later by t £^enn“hght tended to 

pressing the bar to , tali *°™er, ^ 
depress the rate of bar press g Pmure 7 5 It is especially 

fered for the different groups, as is shown in Figure P 
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ment value of stimuli associated with shock termination with a somewhat 
different procedure. They trained rats to run a gray straight alley to 
reach a distinctive (black on some trials and white on others) goal alley 
and, eventually, food. On some trials, the straight alley floor was covered 
with sponge rubber and terminated in one goal alley. On other trials the 
floor was an electrically charged grid and led to the other goal alley. For 
half of the rats the white alley followed the grid; for the other half, the 
black alley followed the grid. After 36 trials of such training the rats 
were tested m a discrimination situation with black as the positive stimu- 
lus and food as reinforcement. This meant that the animals originally 
trained with the black alley following shock had, as the positive stimulus, 
the eo or previously associated with shock cessation. If such stimuli have 
secondary reinforcing value, these animals should learn the discrimina- 
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testin/eondition, 011 vT ^ expectatlon - A veraging across different 

the black aUe^M^ng ItaT TT* ^ 

age nuMbees of errors made by the two groups w“e 3 0 “nd « 

elated wTttfte ctsatlon ' ^ t0 tiat stimuli asso- 

another way, thej^take'on a^v^ue^eeminelvt^ 1 ^ L “° M * 

by stimuli which accompany shock onset Y 6 ° PP ° Slte of that acquired 
fore, arises : Do stinj 

(positive or negative) always acquire termmat, °“ »f any TICS 

duced when they are paired with UCS-onset ! The ‘r Pr °' 

remforcers is even scantier than for npo- +• ’ . 6 evidenee for positive 
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way as it controls the level of learning Several studies have been per- 
formed to test each of these hypotheses 


Secondary Reinforcement and the Number 
of Primary Reinforcements 

The most obvious learning variable controlling the amount of secondary 
reinforcement is the number of pairings of the originally neutral stimuli 
with primary reinforcement On the assumption that the development of 
secondary remfoicement is influenced by the same factors as other learn* 
mg, increasing numbers of such pairings should lead to increasing 
amounts of secondary reinforcement There are three large scale studies 
of this variable, each employing a somewhat different apparatus and 
introducing important procedural variations, yet the results hate been 
in general agreement In each experiment (Bersh, 1951, Hall, 1951, 
Miles, 1956) secondary reinforcement lias been found to be an increasing 
function of the number of pairings of the secondary reinforcing stimulus 
with the primary reinforcer 

One of these studies, that of Hall (1951), employed a procedure which 
was almost exactly the same as that of Saltzman, described previously 
Hall ran 72 rats, after 22 hours of water deprivation (22 hours thirsty), 
in a straight alley with water as a reinforcement The animals received 
8 trials a day On 5 of these trials the goal box was a distinctive color 
(white for some subjects, black for others) and contained water On the 
other three trials the goal box was the other color and was empty Note 
that tins is essentially Saltzman 's differential reinforcement procedure 

There were three subgroups of animals in the experiment, differing in 
the number of secondary reinforcement trials they received in this phase 
of the experiment One subgroup received 25 such reinforcements, an 
other received 50, a third received 75 Following this training the rats 
received 15 test trials m a T maze, with the previously rtinforccd goal 
box at the end of one arm of the T and the previously non reinforced box 
at the other The average numbers of responses leading to the positive 
goal box were 10 7, 11 5, and 12 J, respectively, for the groups receiving 
25, 50, and 75 reinforcements m the training series, indicating that the 
strength of secondary reinforcement depends, as expected, upon the nura 
her of acquisition trials It is reassuring to note that the measure for the 
25 trial group is exactly the same as that obtained by Saltzman for Ins 
comparable group Bcrsh’s (1951) study was very unlike Hall s in that 
Bersh used a free responding situation, the Skinner box, and a diffirent 
method In Bersh 's study the secondary reinforcing stimulus was a light, 
produced by pressing the bar m the Skinner box, and, therefore, paired 
with food There were six different groups of rats in the experiment, ap- 
proximately equated for response rate without reinforcement (aprranf 
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level). One group was a control group which received 120 reinforcements 
without the light. The other 5 groups received 10, 20, 40, 80, or 120 pair- 
ings of light and food. Specifically, the response of pressing the bar pro- 
duced a light of three-seconds duration, with food delivered at the end of 
the first second. Thus, food followed the onset of the potential secondary 
reinforcer by one second, a condition which we will later see is a favorable 
one for the establishment of secondary reinforcement. 

The plan of the experiment was to extinguish all groups with a pro- 
cedure in which the bar press produced the light, and then to compare the 
number of responses to extinction of the various groups. It is clear, how- 
ever, that the different numbers of prior primary reinforcements for bar 
pressing would produce different numbers of responses to extinction with- 
out involving the function of secondary reinforcement. For this reason 
the rats were first extinguished without the secondary reinforcer to bring 
them all to the same level. Following this there was a final testing phase 
of the experiment in which the response of pressing the bar produced a 
light of one-seeond duration. The median numbers of responses for each 
group, obtained in two 45-minute test sessions separated by one day, ap- 
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food pairings in a way which resembles a typical learning curve. The fact 
that secondary reinforcing stimulus retarded extinction, even in the zero- 
trial group, is possibly the result of the fact that dim lights are innately 
mildly reinforcing (Kish, 1955). 


The Extinction of Secondary Reinforcement 

As would he expected, when a secondary reinforcer is presented alone, 
without the primary reinforcer, it loses its secondary reinforcing power. 
Demonstrations of such extinction of secondary reinforcement are nu- 
merous. Pavlov encountered it as a difficulty in the establishment of 
higher-order conditioned responses. He also often deliberately extin- 
guished the “natural” salivary reflex to the sight of food for experi- 
mental purposes. This demonstrates the interesting and important inci- 
dental point that the sight of food is a secondary, rather than a primary, 
reinf oreer. Grindley, in an early investigation of the amount of reinforce- 
ment (1929), ran one group of chicks to food under a glass cover, which 
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made it visible but unavailable He found that, after a short acquisition 
period, the chickens ceased to run under this condition (see Figure 7 8) 
One possible interpretation is in terms of the extinction of secondary re- 
inforcement Saltzman, in the study already described, found that sec 
ondary reinforcement tended to extinguish after a few trials without 
the primary reinforcer, particularly if it had been established by continu- 
ous reinforcement Grindley’s results seem to reflect exactly the same 
sort of process 

Miles (1956b) has performed an experiment which provides a cunc 
revealing, in a preliminary way, the course of extinction of secondary 
reinforcement As m his prnious (1956a) study, Miles trained rats to 
press the bar in the Skinner box to obtain food, the administration of 
which was accompanied by a combination of light and click For extinc 
tion of the bar pressing response the rats were divided into two groups, 
one of which was extinguished with, and one without, the secondary rc 
mforcers To balance out possible innate differences w rate of responding, 
the conditions were switched for the two groups from day to day On one 
day the secondary reinforcers followed the response, on the next daj 
they did not Proceeding in this manner, Miles administered a scries of 
hour long extinction sessions For each session, the difference between the 
numbers of responses in the two conditions provides a measure of the 
strength of secondary reinforcement When these differences are plotted 
as a function of the extinction session, the resulting curie is a decreasing 
ogive shown in Figure 7 9 with \ery little extinction of secondary rein- 
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Another demonstration of the extinction of secondary reinforcement, 
detailed discussion of which seems more appropriately postponed until 
we discuss extinction in general (Chap. 10), comes from studies of latent 
extinction. In these experiments all subjects (usually rats) learn some 
instrumental response, such as running a maze. Then, prior to extinction, 
an experimental group is placed in the goal box without primary rein- 
forcement for a time. The control group is placed in some unfamiliar 
P ace also without primary reinforcement. The test of the influence of 

stattf "n T f 1 T dm : e iS in a S6rieS ° f estinctio11 trials. The demon- 
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liad used in their efforts to demonstrate secondary reinforcement, and to 
compare their procedure with other, successful ones. The one condition 
which was clearly different in their study was in the temporal relation of 
the light to the food Specifically, their procedure had been to wait, 
after the rat pressed the bar, until it seized the food and began eating; 
then the light was turned on This meant that the light came on after the 
food, and that the procedure could be looked upon as a case of backward 
conditioning, which produces little, if any, learning in the typical classical 
conditioning experiment. It will also be recalled that Pavlov and Honor* 
ski both claim that a stimulus presented during eating acquires strong 
inhibitory properties. 

Pursuing this suggestion as a second part of the study already de- 
scribed in connection with the number of primary reinforcements, Bersh 
(1951) first paired a light with food in the Skinner situation, with the 
bar temporarily removed. For different groups of subjects, a pellet of 
food appeared 00, 0 5, 10, 2 0, 4 0, or 10 0 seconds after the onset of 
light, on a total of 160 trials. Following this procedure, the bar was made 
available and conditions were arranged so that a bar depression produced 
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functions obtained in classical conditioning is quite apparent. At the 
same time, substantial amounts of secondary reinforcement were obtained 
at intervals considerably longer than one would think possible from 

comparable studies of the interstimulus interval in regular classical 
conditioning. 

Similar results were obtained in an experiment by Jenkins' (1950). 

mvohed six groups of rats and a procedure much like that of 
el- . The secondary reinforcing stimulus, a buzzer sounding for three 
seconds, was presented to rats in a Skinner box on 100 trials to 5 of the 6 
groups. Food followed the termination of the buzzer bv 1 3 9 27 or 
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Secondary Reinforcement and the Amount 
of Primary Reward 
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A third investigation which shows that secondary reinforcement de- 
pends upon the amount of primary reinforcement approached the prob- 
lem m a somewhat different way, by varying the amount of eating time 
allowed m different goal boxes (Powell and Perkins, 1957). In this study 

f f t0 10 f reward in Preliminary training designed 
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primary remforcer would lead to tile entablement of secondary rem- 
forcemeat that is especially durable Although several experiments of tins 
sort have been done, none of them has employed what seems to be the 
most appropriate procedure Such an ideal experiment would begin by 
providing some neutral stimulus with secondary reinforcement power in 
a training situation uncontaminated by the presence of the response later 
to be used to test for the strength of secondary reinforcement During 
this initial training, there would be two groups For one group the pri- 
marily reinforcing stimulus would follow the neutral stimulus on c\ery 
trial For the other group, the primarily reinforcing stimulus would 
follow the presentations of the neutral stimulus only on some predetcr 
mined fraction of the training trials Then both groups would bo tested 
by putting them into a situation where a certain response produced the 
(now) secondary remforcer Under these conditions the group which had 
been trained with an intermittent schedule should learn better and con- 
tinue responding longer than the group trained with a continuous 
schedule 


In the studies which have been done on this problem (Saltzman, 11149, 
Dinsmoor, 1952, Melchmg, 2954, Clayton, 195C) the response ultimately 
to be used in the test situation has also been required of the animal during 
initial secondary reinforcement training Melchxng, for example, used 
rats m a Skinner box Following habituation and routine pretraming, 
they were gi\en 20 reinforcements for bar pressing For two subgroups, 
every response produced the sound of a buzzer (secondary remforcer) , 
for two other subgroups, the buzzer appeared haphazard!} following half 
of the bar presses, for a third pair of subgroups, the buzzer never 
sounded Following this training the rats were extinguished One sub- 
group from each main experimental condition was extinguished with the 
buzzer sounding after each response , the other sub group was extinguished 
without it There were no differences in resistance to extinction which 
could be attributed to the greater secondary reinforcing \alue of the buz 
zer for the two 50 per cent groups than for the two 300 per cent groups. 
In Dinsmoor’s experiment, the comparison was between the rciuforcmg 
value of a light when it was used as the signal for the beginning of a three- 
minute fixed interval schedule or as a signal that the next response would 
bo rewarded In later extinction tests for the effectiveness of the light as 
a secondary remforcer, it was found that the light contributed more to 
resistance to extinction for the three minute reinforcement group, on 
outcome which might mean that the strength of seconder} reinforcement 
is increased if it is established by means of an intermittent schedule 
It should bo noted, however, that in both of these experiments, the com 
parison is complicated b} the primary reinforcement condition* during 
training In Stelching's experiment all of the animals were reinforrnl 
primarily on all trials. It seems quite hid} that resistance to extinction 
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would be similar for all groups and that the difference between a con- 
tinuous and partial schedule of secondary reinforcement would have little 
effect. In Dmsmoor’s experiment, the original schedules of primary rein- 
forcement produce very different resistances to extinction in their own 
right and in the same direction as the differences attributed to the sched- 
ule on which the neutral stimulus and primary reinforcer were paired. 
This makes the secondary reinforcement values following the two sched- 
ules very difficult to assess. 

** “ 1 more a demonstration than an experiment, Zimmerman 

menf h i n ° f deVel ° pin = ver y strong secondary reinforce- 

ment by gradually decreasing the probability that primary reinforcement 
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Secondary Reinforcement and Drive 

«1) that the level Kereises™™^™ f"] la * er (PP ' 
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DEPRIVATION 

FIGURE 7 12 The strength of secondary reinforcement based upon primary 
food reward as a function of the number of hours of food deprivation (Miles. 
Z9S6a) 


operating and (2) 'when all motives under the control of the experimenter 
are minimized The evidence on the first of these specific questions is con 
sistent As with other learning secondarily reinforced behavior is mam 
tamed when the only drive is an irrelevant one Estes (1349) studied the 
bar pressing behavior of three groups of rats reinforced hi the sound of 
a stimulus previously associated with the presentation of water for two of 
the groups The third group was a control group which had had no sec 
ondary reinforcement training In the test the rats were satiated for 
water but 6 or 23 hours hungry The hungrier group produced more re 
spouses than either the six hour group or the control group indicating 
that a secondary reinforcer will exert its reinforcing influence upon he 
havior in the absence of the original drive The six hour hunger group 
however did not differ from the control group suggesting that a fairly 
strong irrelevant drive is necessary for the effect to appear 

Tins last result raises the question in its other form Does a secondary 
reinforcer exert its reinforcing influence upon behavior when all drives 
are minimally lowt Estes' experiment suggests that they do not Seldom 
berg and Pratt (19oG) offer the same interpretation on the basis of (ho 
results of a T maze experiment in which rats were secondarily reinforced 
by the sight and smell of inaccessible food Satiated rats learned nothing 
but hungry rats learned to take the path leading to the secondar) rein 
forcer Wike and Casey (19j4b) have obtained dnta in support of the 
opposite conclusion m an experiment cm pi 03 mg a straight alley api ara 
tus Having previously found 111 agreement with Estes tint the si„ht of 
food is reinforcing for thirst) rats satiated for food (Hike and Cimv 
1934a) » they did another study with rata satiated for both food and 
water Iheir results showed that such satiated animals ran foster if they 
were rewarded by reaching a goal box containing fixxl than if tiny f un 1 
an empty goal box at the end of the alley Seward and I*cvy (I-hil) laie 
also obtained data in support of the conclusion that a secondary rrm 
forcer will support the behavior of satiated rats. 
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These discrepant results seem to require a methodological comment: 
e on y evidence so far proposed to demonstrate the existence of a sec- 
ondary remforcer depends upon the ability of secondary reinforcement 

In^thc eh r°? r *° Pr ° Vide fte baSis ,or Iearni ”S a response, 
harior „r 6 ’ dem ° nstrati “ upon the occurrence of be- 

beharior -t,T, SeconclaI7 reinforcement effect is possible. The 
ones Xch^m , “"T 7 eaperiment this problem, however, are 
moti^Hon T 7 WOuU n0t oceur “ the complete absence of 
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to consider in attei^ti eU T Slt7 i S6SUal ones are obvi °us possibilities 
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somewhat different tJL TC 1 qUeshon under investigation takes a 
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tion 0 f the effect of ZeZ^ZT^ ^ ^ ^ to 
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tion of secondaxy reinf or^tt affe^T ^ ^ tbe W ° f tbe acquit 
foreer? Experimentally this question 6 ?° Wer of tbe secondary rein- 
subjects learn the secondary reinforcp^^ 8 that different groups of 
and demonstrate its magnitude with d C Under ^“g drive levels 
ee ure as m Ins previous study (Hall Ci Using tbe same Pro- 
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remforcer later. Rats, either 6- or 2o\C° n + l Value of tb e secondary 
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the secondary reinforcing status of the goal box. There ^ hoover no 
difference between the groups winch had been teamed unde r G i and 1- 
htmrs of deprivation Averaged across all subgroups the number of 

other forms of learning. 

EFFECTS OF SECONDARY REINFORCEMENT 

TTgen™ properties of secondary 
( 1951 ), are the same as those of prim^ (2) [ t „ m support tl.e 

inforcer can be used to pro provides the organism with a 

performance of a learned respo , possesses an incentive func- 

means of bridging gaps in tunc a reinforcer will lead an organism to 
tion in that the sight of a seeonda^ J ncgat iv . re.nforeer, to 

strive to obtain it, or, in the case o scc „pdary reinforcement 

avoid it. Some of the evidence > f " this chapter In addition, 

these properties has been presents s-Il theorists to 

the concept of secondary “^phenomena. The .Cons 

InWnnllWnt RS .!™”* 1 “ d 

Several investigators have i° t0 performance in partial 

forcement malics an important sUch c j pcr imciits the non-rein- 

reinforcement learning espcrtm 'forced because tbc typical procedure 
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firmed this expectation. The 100 per cent group made 8.5 correct choices 
m 11 trials, whereas the 50 per cent group made only 6.0. The difference 
of 2.5 correct responses was highly reliable in statistical terms. 

Other evidence that the so-called non-reinforeed responses in a partial 
reinforcement schedule actually are reinforced comes from studies in 
which the number of primarily reinforced trials remains the same for 
several groups of subjects which differ in the number of non-reinforced 
(that is, secondarily reinforced) trials. These studies (Notterman 1951: 

enny and Dunham, 1951) have shown that the greater the number of 
uon-reinforced trials, the better is the subject’s performance. 


Discrimination Learning and Secondary Reinforcement 

aittnSv^ inat i° n - ning ’ the animal 18 required t0 choose between 
ternative stimuli and is rewarded if it chooses correctly Within such 

pe” e th; fir C s°t ndary . r f t f0rcement has tW0 mai * opportunities to 

st r “ r ponses lead to 

forcer in one and not In thither t ° f & Primary ^ 

ondarily reinforced. This should make suchTd ' r 
cult to master than one in which the f .™ natl0n more diffi " 

Studies with the T-maze (Denny 1948 cfu Sltuatl0ns are Afferent. 

black-white discrimination problem 1 (Ehre^f ^a’ i« 50) ^ ™ th & 
yielded results in support of^such^n ^hrenfreund, 1949, 1954) have 

called attention to the most obvious interW* ^ lrenfr J; und > for example, 
troduced into many two-choice 1 as, • • rin " secondary reinforcer in- 

the goal box following an incorre r ^ mg Sltuatlous > the empty food dish in 
general sort make the response whioh eSP ° USe ' Sin ° e m ° St studies of this 

rect for another, thus balancing out naturalT a* SUbgroup incor * 
dommantly in one way or the other iff deM ies to respond pre- 
food dishes in both goal boxes and to’n t f S °ff tlmes convenient to have 
for the animal being run. As Ehre f put food 111 the one which is correct 
that a very strong secondary reinforceTk V ° mtS 0Ut ’ howe ver, this means 
response. To test the validity of such ? * t0 reward an incorrect 

a black-white discrimination experimenT 01 *- 11 ^ ^brenfreund performed 
only procedural difference was that fn p US1Ug tW ° groups of rats. The 
forcement group) there was never ^’a food ^ 1 (& "Secondary rein- 
whereas for Group 2 (a secondary re inf ^ 1U the negative goal box, 
similar to the one in the positive goal box ^ “ ent grou P) a food cup 
tjve goal box. The effect produced I WayS present in the nega- 
shown m Figure 7.13. Clearly the presenc S f“ lngly SmaP variation is 
Portan ^ffereue 6 and led tQ & m ce ° f the food cup made an im- 

the discrimination developed For the ^ “ tbe Speed with 

r the non-seeondary reinforce- 
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ment group 24 4 taafe and 7 2 errors were required to attain a mlenon 
Of 90 per cent correct responses For the secondary reinforcement group 
the comparable values were 42 2 trials and 17 5 errors 1 

The second opportunity for secondary reinforcement to become in 
voived in discrimination learning occurs if the cue identifying the correct 



FIGURE 7 13 The effect ol a secondary reintorcer upon ihe progr«« of dis- 
crimination learning For group It a food cup like that used to contain re- 
ward on correct trials was present in the negative goal box. For group I there 
was never a food cup in the negative goal box. The relatively slower learning 
of group II is attributed to the fact that the incorrect responses received 
secondary reinforcement (Ehrenfreund, 1949) 

choice in a discrimination situation has some of the same properties as 
the goal to which it leads In such aa event, the positive stimulus vrould 
be a secondary remforcer and attract behavior through its incentive 
function One of the most important hinds of evidence in favor of this 
interpretation has already been presented (p 133) in connection with 
the delaj of reinforcement gradient This vvas Grice s (IDJsb) dimon 
stration that miking the correct and incorrect choice alios the same 
color as the positive and negative goal boxo-» in a black white discrimina 
tion enables the animal to learn for a dclj>cd reward This result lias 
been confirmed b> Bauer ind Lawrence ( 19o I) and bj Webb and Nolan 
(1933) More recently both Levy (I9a7) and Grice and Goldman <19 '3) 
have analyzed this effect further and lnvo obtained good evidence that 
two kinds of second arj reinforcement arc involved powmc he evudary 
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firmed this expectation. The 100 per cent group made 8.5 correct choices 
in 11 trials, whereas the 50 per cent group made only 6.0. The difference 
of 2.5 correct responses was highly reliable in statistical terms. 

Other evidence that the so-called non-reinforeed responses in a partial 
reinforcement schedule actually are reinforced comes from studies in 
which the number of primarily reinforced trials remains the same for 
several groups of subjects which differ in the number of non-reinforced 
(that is, secondarily reinforced) trials. These studies (Notterman, 1951 ; 
Denny and Dunham, 1951) have shown that the greater the number of 
non-reinforced trials, the better is the subject’s performance. 


Discrimination Learning and Secondary Reinforcement 

In discrimination learning, the animal is required to choose between 
alternative stimuli and is rewarded if it chooses correctly. Within such 
an experiment, secondary reinforcement has two main opportunities to 
operate. In the first ease, if the correct and incorrect responses lead to 
similar goal situations, differing only in the presence of a primary rein- 
forcer in one and not in the other, the incorrect response is actually sec- 
ondarily reinforced. This should make such a discrimination more diffi- 
cult to master than one in which the two goal situations are different. 
Studies with the T-maze (Denny, 1948; Saltzman, 1950) and with a 
black-white discrimination problem (Ehrenfreund, 1949, 1954) have 
yielded results in support of such reasoning. Ehrenfreund, for example, 
called attention to the most obvious interfering secondary reinforcer in- 
troduced into many two-choice learning situations, the empty food dish in 
e O oa ox o oving an incorrect response. Since most studies of this 

rect for^ntT IT 0 ” 5 ' is correet for one subgroup incor- 

domiuaotlv CT ’ S out tendencies to respond pre- 

foTa aSt I “hTh^ T ' ° tier ’ iS SOmet ™ es convenient to have 
or h. g rr n4 t0 P,lt lood “ one which is correet 

ftat a v?r s t™“ S Ehrenlre '“ d out. however, this means 

relJe^o ^ test th?° n , ^ r< f l0rcer pres “‘ toward an incorrect 

a black-white discrimination enp^rimentT”"’ , Ehrenfremid P eriormed 
only procedural difference was that (or Qrou S w ST °' 1PS ^ T 
forcement group) there was po™ ’ T ? P 1 (a non - s eeondary rem- 
whereas fo°r Group 2 (a seeoXrv" ^ » egative * oal b ° S ’ 

similar to the one in the positive reinforcement group) a food cup 

tive goal box. The effect produced^y 0 ^ present in tlle ne - a ' 

shown in Figure 7 19 ,, y ^ ais seem mgly small variation is 

portant difference and led L a e J- Te * ei * ee ° f tbe food CU P made an im- 

Which the discr4“Lf dXTC:r ati011 ” the Sp “ d ^ 

P • or the non-secondary reinforce- 



I* « It- l« tf 2* Si* 16* 41* 44 oF 5* «l 

3 10 IS 20 25 SO 35 40 45 SO S3 SO <3 

TRIALS 

FIGURE 7 13 The effect of a secondary reinforcer upon the progress of di8 
crimination learning For group II a food cup like that used to contain re 
ward on correct trials was present in the negative goal box For group I there 
was never a food cup m the negative goal box. The relatively slower learning 
of group II is attributed to the fact that the incorrect responses received 
secondary reinforcement (Ehrenfreund, 1949} 

choice in a discrimination situation has some of the same properties as 
the goal to -which it leads In such an event the positive stimulus would 
be a secondary reinforcer and attract behavior through its incentive 
function One of the most important kinds of evidence in faior of this 
interpretation has already been presented (p 153) in connection with 
the delay of reinforcement gradient This was Grice s (1948b) demon 
stration that making the correct and incorrect choice alleys the same 
color as the positive and negative goal boxes in a black white discrimina 
tion enables the animal to learn for a delayed reward This result has 
been confirmed by Bauer and Lawrence (1953) and by Webb and Nolan 
(1953) More recently both Levy (1957) and Grice and Goldman (195o) 
have analyzed this effect further and have obtained good e\idence that 
two kinds of secondary reinforcement are involved positive secondary 
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reinforcement generalizing from the reinforced trials in the distinctively 
colored positive goal situation, and negative secondary reinforcement 
generalizing from the non-reinforced trials in the other goal box. 

To make this demonstration, Grice and Goldman ran four groups of 
rats in a black-white discrimination in which white was always positive 
and a two-second delay followed the choice. The four groups were run 
under the following conditions: For one group (WB, that is, White- 
Black) the two end boxes in the discrimination apparatus were painted 
white and black in correspondence with the correct and incorrect choices. 
For a second group (GG) both end boxes were gray. For the remaining 
two groups one of the end boxes matched the color of one of the choices 
and the other was gray. For Group WG, the positive goal box was white ; 
and, for Group GB the negative goal box was black. Median numbers of 
errors required to master the black-white discrimination were : For Group 
GG, 128 ; for Group WB, 43 ; for Group WG, 58 ; and for Group GB, 40. 
here were important differences only between Group GG and the others, 
nee and Goldman take this to mean that secondary reinforcement (posi- 
lve or negative) established to the end box color generalizes to the choice 

S;, a vt the animal t0 ap P roach or a ™d the cue in question. 

about as hen eft* * f ma ^ nltl ^ es the effects obtained that one process is 
beneficial as the other to the formation of the discrimination. 

R“r n f t Intermittent Schedules o£ 

oiary“r^^ far ' the ** the strength of see- 

t ■ secondary has 

ment, it is to be expTeteTth^ “f * f ° f 

lead to the establishment of more SChedule ' r ° uM 

schedule of secondary reinforcement A • \ behavior than a continuous 

tion (Dinsmoor, Ki^nf Ser 1%^,^ “ X** *“* 
obtained negative results There u ’ ^ layton » 1956) have usually 
with lower animals (Zimmerman ^ SUCCessful demonstration 

numan guessing behavior In the C ’ 1 * 4 . ’ &nd a number of others on 
done by Humphreys (1939b Wh 1 Studies ’ the firs t of which was 
verbal analogue to cla pr0videS a 

sides of a large board serve as “CS” and^UG^” 0n ^ tW ° 

appears, the subject is required tn , d x. U ° S * When the CS 
will come on also. By varying the g T J! bether or not the UCS light 
come on, it is possible to set up ant ? that tbe UCS ^ wiU 

typical set of results obtained with t? S f hedule of reinforcement. A 
In this study (Grant, Hake, and Ho^etT mT" * FigUre 7 - 14 ’ 
reement (teals when the seoond ^‘ages^of re- 
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FIGURE 7.14. The effect of percentage of reinforcement upon the acquisition 
and extinction of a verbal guessing habit (Grant, Hake, and Hornseih, 1951). 

and 100. Two important facts stand out: (1) The percentage of guesses 
that the UCS light will come on begins at about 50 per cent and changes 
gradually until it matches the actual probability fairly closely. This same 
result has been obtained repeatedly (Estes and Straughan, 1954; Estes, 
1957). (2) In extinction, the order of increasing resistance to extinction 
is approximately the reverse of percentage of reinforcement in training. 
The 25 per cent group is most resistant ; the 100 per cent group is least 
resistant. This result, too, has been obtained a number of times (Hum- 
phreys, 1939; Grant, Hornseth, and Hake, 1950). It, of course, corres- 
ponds very well to the result occurring in the typical animal experiment 
For our purposes, the important point is that the reinforcer employed 
in this study is a secondary one. The subject has no innate capacity to be 
reinforced by the appearance of a light. Any doubt on this point is dis- 
pelled by the demonstration (Grant, Hake, and Hornseth, 1951) that, 
with the instructions appropriately altered, the light's failure to come on 
is just as good a reinforcement as its appearance The fact that such a 
remforcer produces more persistent behavior when applied on a partial 
schedule than when it is applied continuously shows that it shares this 
characteristic with other reinforcers 

On the other hand the verbal conditioning situation differs from other 
learning situations in a number of ways: (1) Other forms of learning do 
not usually lead the subject to respond in proportion to the probability 
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of reinforcement. (2) The verbal conditioning situation is not influenced 
by distribution of practice as most forms of learning are (Humphreys, 
Miller, and Ellson, 1940; Grant, Hornseth, and Hake, 1950). For such 
reasons, interest seems gradually to have shifted away from this experi- 
ment as an analogue of simple learning. More and more it is being dealt 
with as something of interest in its own right as an elementary form of 
predictive or decision-making behavior (Estes, 1954, 1955). 


The Generalization of Secondary Reinforcement across 
Drives 

An important segment of the literature on secondary reinforcement has 
been concerned with the extent to which it generalizes from one drive to 
another We have already seen in connection with the studies of Estes 
and Wike and Casey that such generalization apparently can occur. The 
results of a second sflidy led Estes (1949) to the conclusion that such 
transfer from one drive to another may result in the loss of as little as 

nhX r Ce ? Z f ® ValUC ° f a secondary reinforcer. In a complete and 

Horn onTdrf r Pe T nt ° f secondary reinforcement 

from one drive to another, D ’Amato (1955b) used a procedure very 

S ™1 box°as a s 2111 ^ 8 reinf °rcement method to establish the 
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very similar to that reported by D'Amato Th,^ °1 tr “ Ia ’ “ Tal " C 
ception (Reid and Slivinski 19541 +>, Us ’ Wltl1 an occasional ex- 

the transferability of secondly 

bS? Reiaf °™“ on, and the Mediation of Gaps 

assigned by Miller to see- 

-=h otherwise weald be 
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of secondary reinforcement are to be found in the delay of re, " £ ' ,rc ' nl “ t 
studies previously discussed (pp 152) where * ~ 
the presence of a secondary reinforcer may enable tb 

2S=Ssa«~S£S 

ning process, but also because it ea nature of secondary 

hopefully, therefore a better understanding of) the natu^^ 

reinforcement The ■ ^"that any response subsequent to 

(1938, pp 52 55, 102 108) sugg nrecedmg response sec 

another m a chain of behavior reinforcement to a matter of 

ondanly This, in M through which see 
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individual pattern of responding which includes the other three feet as well. As 
soon as the buzz spurs him to action on any trial he may, for example, (<z) raise 
the right-front foot and imm ediately return it to its original position, (b) raise 
the left-rear foot and replace it at once also, (c) raise and replace the left-front 
foot, and finally (d) lift and hold up the right-rear (or shocked) member during 
the time when the shock would have occurred had he failed to respond. Almost 
any other reaction or combination of reactions . . . may come to be uniformly 

repeated as a preliminary to the flexing of the shocked limb itself. (Kellogg, 
1949, p. 173) V 


. Probab ly the behavior of the rat in the usual delay-of-reinforeement 
situation is somewhat different from this, consisting in the maintenance 
of a fixed orientation toward the goal; and the greatest delay of rein- 
forcement with which the animal can learn is simply the length of time 
it can hold such a response. Carlton (1955) performed an experiment 
which produced results quite consistent with this interpretation. Rats 
learned to press a bar in the Skinner box to get food which was delivered 

Eaoh r ofT 7 ° r " t6r 10 SeC ° ndS f ° r tW ° mai * ^oups of subjects. 
Each of these main groups consisted of two subgroups for which the delav 

chambers differed. For one group, the confined group 

the delay chamber was small, 6 inches * 3 inches x 2 nches For another 

group, the unconfined group (UC, in Figure 7.15), it was launches 
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x 17 inches x 11# inches. The purpose of introducing this difference was 
to test the hypothesis that the animals in the confined group, with less 
opportunity to produce interfering responses, would maintain their goal 
orientation during the delay period better than the unconfined group, and 
would learn better with the 10-second delay of reinforcement. To provide 
a more general test of the hypothesis, the delay conditions were reversed 
after 60 trials: those animals originally receiving food immediately were 
shifted to a 10-second delay and vice versa. The results of the experiment 
appearing in Figure 7.15 confirm the hypothesis in detail. Observe that, 
for subjects receiving immediate reinforcement, the confinement condi- 
tion was unimportant and both groups learned better than the delay 
groups. For these latter groups the confinement condition makes the 
expected difference. The performance of the confined group was con- 
siderably better than that of the unconfined group. 3 


Related Phenomena in Classical Conditioning 

In classical conditioning studies the long trace and long delay pro- 
cedures create problems which are the same as those deriving from long 
delays of reinforcement in instrumental conditioning In these experi- 
ments it is less likely, however, that overt motor responses are critical in 
mediating the delay. It may be that recent studies of the electroence- 
phalographic correlates of conditioning provide an important lead in the 
direction of an explanation Motokawa and his associates (Motokawa, 
1949 j Motokawa and Huzimori, 1949; Iwama, 1950; Iwama and Abe, 
3952, 1953) have reported a series of interesting investigations in which 
autonomic and electroencephalographic responses were conditioned simul- 
taneously, the main interest being in parallels between the two. The EEG 
response employed (“excitation potential” or “e. p.”) is a complicated 
one consisting, in part, of the blocking of the alpha rhythm which had 
previously been shown to be conditionable (Jasper and Skagass, 1941a) 4 

* This interpretation of the effects of delay stresses just one of their aspects. It will 
he recognized, however, that delays have other effects such as those of producing 
"frustration” and "emotional upset.” W L Brown and his associates have con- 
sidered these influences in interpreting the effects of intramaze delay (Brown, Gentry, 
and Kaplan, 1948; Brown and Gentry, 1948; Gentry, Brown, and Kaplan, 1949; 
Brown, Gentry, and Bosworth, 1949; Brown and Sjoberg, 1950 ; Brown and Dalrymple, 
1951) These concepts have also loomed large in discussions of extinction (pp 309 to 
317). 

4 The conditioning of the EEG and EEG correlates of other conditioning are be 
coming matters of increasing interest General discussion of these topics are to be 
found in the transactions of Ihe 1958 and 1959 Josiah Macy Jr. Foundation confer- 
ences on "The Central Ner^ts System and Behavior" (Brazier, 1959a, 1959b). 
These discussions include references to relevant literature. See also Ellingson (19o6)» 
Jasper and Shagasa (1941a, 1941b), Knott and Henry (1941), Knott, Platt, and 
Hadley (1944), Morrell and Jasper (1956), Morrell, Roberta, and Jasper (1956), and 
Shagass and Johnson (1943). 



200 CONDITIONING AND LEARNING 

In these studies, a conditioned excitation potential was shown to develop 
along with the conditioned GSR and seemed to be essential to the occur- 
rence of the latter response. The evidence for this interpretation is fairly 
impressive: (1) The excitation potential occurs earlier in the interstimu- 
lus interval than the GSR. (2) It appears earlier in the course of condi- 
tioning than the GSR. (3) The excitation potential occasional! y occurs 

* bsene ® the GSR ’ but the reverse apparently never happens. 
(■A) When a GSR reappears, as sometimes happens after it has been ex- 
tmgmshed, the reappearance is always associated with the occurrence of 
an EEG reaction, which also extinguishes with non-reinforcement. The 
concept of the excitation potential as providing a mediating link which 
bridges long mterstimulus intervals, thus, is at least tenable 
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The Nature of Secondary Reinforcement 
The conditions necessary to establish secondary reinforcement are the 
same as those employed in classical conditioning A“ 

stimulus is paired with a primary remforcer As a result, this stimulus ac 
quires the capacity to function much as the primary reinforcer ongmaUy 

t— o/of a secondary reinforcer will lead the 
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experiment remains to be done. An additional variable, which has no 
effect, is drive level at tlie time of the establishment of secondary rein- 
forcement. This result is consistent with those obtained in experiments 
on ordinary learning. 


Effects of Secondary Reinforcement on Behavior 

The concept of secondary reinforcement has been employed by S-R 
theorists to explain a variety of experimental results. It has been shown, 
for example, that the effect of partial reinforcement in increasing resis- 
tance to extinction is partly a matter of secondary reinforcement which 
occurs on otherwise non-reinxorced trials during training. Similarly, it 
has been demonstrated that discrimination learning benefits if the posi- 
tive and negative stimuli to be discr imin ated somehow match the rein- 
forced and non-reinforeed goals. In related experimental tests it has been 
shown that secondary reinforcement generalizes across drives. 

One of the most important influences of secondary reinforcement is in 
its ability to assist the organism in the mediation of gaps in time. There 
is good reason to believe that this influence occurs in some cases as a 
result of chains of “superstitious” behavior elicited by the secondary 
remforeer. In other cases the responses may be implicit internal reac- 
tions, conceivably of the central nervous system. 
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General Theories of 
Reinforcement 


The previous chapters have reviewed a considerable amount of material 
which shows that the empirical operation of reinforcement is essential 
to learning In this empirical sense, *a reinforcer is an event which, » 
employed appropriately, increases the probability of occurrence of a 
response m a learning situation Without exception these events are 
Stimulus changes such as accompany presenting food or water, turning 
a shock on or turning it off, or presenting a stimulus that has a history 
of being correlated with such happenings The statement that some such 
occurrence is necessary to learning is a statement of the empirical prin 
ciple of reinforcement or the empirical law of effect (McGeoch and 
Irion, 1952) At this empirical level there is no important dispute about 
the validity of the principle of reinforcement, and there is no such thing 
as a non reinforcement theorist Even Guthrie, who is often described 
as holding a non reinforcement position, maintains such a view only 
with respect to the so called theoretical law of effect, specifically inter 
preted m terms of drive reduction or satisfaction 

As this last statement implies there are many different opinions at the 
theoretical level as to the mechanism through which reinforcement pro 
dunes its effects Moreover, these differences nf opinion exist simnltane 
ously on several dimensions Conceivably the most basic question of all is 
whether the single concept, reinforcement, represents a single process 
Is it possible that a response conditioned by the administration of an 
unavoidable noxious stimulus is reinforced in the same way as a response 
acquired because it avoids the same stimulus f And is reinforcement in 
either of these situations the same as that which occurs with the presen 
tation of food or of a secondary remforcer? We shall want to discuss 
this problem m some detail later In the meantime, it is clear that, 
although all instances of reinforcement are abstractly similar m the 
sense of involving stimulus change, their yanety is so great that manj 
writers ha\e been led to suspect that the underlying processes may not 
always be the same Indirectly, this suspicion has led to the sketching of 
203 
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alternative theories of reinforcement: Different psychologists, drawing 
their empirical evidence from diverse experimental situations, have been 
impressed with various aspects of the reinforcement occasion and on this 
basis have proposed a variety of general theories of reinforcement. 


THE HISTORY OF THE PROBLEM OF 
REINFORCEMENT 

Classically there have been three influential views of re inf orcement 
which have been termed contiguity, effect, and expectancy theories, re- 
spectively. In actuality these positions are better described as hypothetical 
principles of reinforcement and, as such, are merely parts of theories 
because each of them has always been assumed to operate in combination 
with other principles. On the other hand, different early theorists made 
one or another of the processes referred to in these terms bear the main 
explanatory burden in their theories. Hence, the designations contiguity 
theory, effect theory, and expectancy theory were appropriate, at least 
at one time. More recently the differences among theories have become 
blurred, chiefly because some of the expectancy theorists now accept 
effect theory for some forms of learning (Tolman, 1949), and the effect 
theorists have included expectancy-like conceptions in their newer formu- 
ations (Hull, 19o2; Spence, 19o6). As a result, we have now emerged 
from the era m which the dominant themes in the psychology of learn- 
ing derived from these central assumptions. Even so, it is instructive to 
review the early history of reinforcement controversy in order to under- 
stand how it forced a change in the approach to, and even in the defini- 
tion^ * ShaU tteref0re summarize the classical posi- 
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Clearly the contiguity or substitution interpretation is most directly 
applicable to classical conditioning It was fundamental to Pavlovs 
theoretical formulations and, in one form or another has been an integral 
part of many more recent treatments of classical conditioning Even in 
this natural context of classical conditioning however contiguity theory 
encounters a major difficulty that of explaining the form of the eondi 
tioned response One of the most difficult problems confronting any 
theory of conditioning is to account for the obsen ed differences between 
a conditioned response and the unconditioned response on which it is 
based The principle of substitution is particularly embarrassed by these 
differences because it demands that the organism act m the presence of 
the conditioned stimulus as though it were in the presence of the uncondi 
tioned stimulus Although something approximating this occasionally 
seems to happen it does not occur often Pavlov (1934) reports that his 
dogs, trained to salivate to a light preceding food sometimes turned 
their heads toward the light and even licked the light globe but this is 
an exception The most common result of classical conditioning as has 
been pointed out previously is that the CR resembles but does not com 
pletely duplicate the UCR The contiguity principle simply formulated 
cannot handle this fact 

Contiguity theory in any form is characterized by the fact that no [ 
additional motivational principle is introduced to account for learning 
Motivation theoretically enters as a factor in determining the occurrence 
and magnitude of the original unconditioned response but the substitu 
tion is dependent merely on the fact of response 

Effect Theory 

Although somewhat similar statements appear in the writings of the 
earlier psychologists the first clear expression of the law of effect is to 
be found in the writings of Thorndike (1911 p 244 See also p 10 of 
this book) Thorndike proposed that responses which have satisfying 
consequences are strengthened and those followed by discomfort or 
annoyance are weakened More recent expressions of this principle differ 
from the original ones chiefly in that other terms (reward reinforcement 
drive reduction ) have been substituted for satisfaction and annojance 
These ehanges have been made in part to avoid certain philosophically 
grounded objections to the original statement 

Since the earlier forms of the law of effect were couched in the Jan 
guage of satisfaction and annoyance they were related historically to 
the pleasure pain theories of the various hedonisms and ran into trouble 
for this reason Most of the hedonistic views localized the experience of 
pain and pleasure m consciousness and therefore made the law of effect 
subjective in a sense to which the temper of the tunes was opposed 
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Because of this, there were numerous attempts to define the concepts 
of satisfaction and annoyance objectively. Thorndike, for example, pro- 
posed an essentially operational approach: “By satisfying state of affairs 
is meant one which the animal does nothing to avoid, often doing such 
things as to attain and preserve it. By a discomforting or annoying 
state of affairs is meant one which the animal commonly avoids and 
abandons” (1911, p. 245). Other definitions emphasized the tension- 
reducing character of reinforcement. It was pointed out that the animal 
in the problem-box or maze is aroused, restless, hypertensed, until the 
food is reached. After ingestion of food, the animal becomes relaxed and 
quiescent. S. Smith and Guthrie (1921) wrote of “maintaining stimuli” 
which are removed by the reinforcing act. In this they followed the con- 
ceptions of prepotent and eonsummatory response proposed earlier by 
Sherrington (1906) and Woodworth (1918). Raup (1925) described the 
end of goal activity to be the reaching of a state of “complacency” simi- 
lar to that which Cannon (1932) has called “homeostasis.” T. L. McCul- 
loch (1939c) proposed that reinforcement may be effective because it 
causes the disappearance of restless, excited behavior. Hull’s (1943) 
drive-reduction theory is similar. 

Other attempts at objectification took a physiological direction and 
attributed the reinforcing effects of satisfies and annoyers to changes in 
resistance to conduction along the appropriate nerve paths (Thorndike, 
1911; Troland, 1928). The line of research barely hinted at in these 
proposals is perhaps the most important one today. As we shall see later, 
many psychologists are looking for the neurological and humoral loci of 

dffertion ° n remf0rCerS and W made infant advances in this 
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that learning occurs because of the action of some outside agent (effect). 
Thus the most conspicuous difference between contiguity and effect ac- 
counts of learning is that the latter contain references to some conse- 
quence of a response, such as “satisfaction,” “drive reduction,” “relax- 
ation,” or “return to homeostasis,” which the latter do not. Compari- 
sons of the two positions have consisted in attempts to determine whether 
these hypothetical aftereffects are, in fact, necessary for learning to occur. 

The difference between the contiguity and effect interpretations of 
learning may he illustrated by the concrete, but hypothetical, example 
in Figure 8.1. In this example it is assumed that a dog is presented with 


Conditioned 

i • response „ . . 

Light ► Knee jerk 

(Conditioned 
stimulus) 


Tendon tap 
(Unconditioned 
stimulus) 



■ Food 
(Reward) 


FIGURE 8.1. Comparison of the principle of contiguity with the law of effect. 
According to the principle of contiguity the conditioned response depends 
upon the occurrence of the unconditioned knee jerk. According to the law 
of effect, the conditioned leg extension depends upon the food reward. For 
further discussion, see text. 


a conditioned stimulus (light), followed by a stimulus which elicits a 
mee jerk, which m turn is followed by a reward (food). According to 
e principle of substitution, the strength of the conditioned response 
(leg extension t° light) depends upon the occurrence and the intensity 
unconditioned knee jerk response. The principle of effect states 

rence all r g i < f lditioned *«*<»»» is a function of the occur- 
rence amount of food, reward. 
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fancies and, therefore, similar behavior sequences will again be evoked 
If the expected consequences do not materialize, the expectancy is weak 
ened and behavior on the next trial may be different Such tentative be 
havior, entered upon m accordance with expected consequences which 
are probable but not certain, will be immediately recognized by the reader 
as familiar in his own experience 

Expectancy theory, as developed by Tolman, is often referred to as a 
non reinforcement theory As in the case of Guthrie’s theory, however, 
tins designation is somewhat misleading The position is a non reinforce 
ment position only in the sense of being a non law of effect position The 
terms sigmficate and goal are the terms referring to reinforcement in 
expectancy theory Learning is assumed to occur whenever a sign is 
followed in time or space (that is, reinforced') by some other signified 
object It should be noted that learning, in expectancy theory, is of a 
“stimulus stimulus’* sort in that the learning organism acquires a 
knowledge of what (stimulus one, or sign) leads to what (stimulus two, 
or sigmficate) in the environment 

In expectancy theory, there are two steps in explanation of any bit 
of learned behavior The first involves the assumption of a sign sigm&cate 
relationship, so that the situation has a new meaning for the organism 
based on its prior experience in it Thus the guinea pigs in the revolving 
cage of Brogden, Lipman, and Culler (1938) must learn that at the sound 
of the buzzer the cage is potentially dangerous (a shock is likely to occur) 
and that the shock is less likely to occur if they start running when the 
buzzer sounds Wolfe’s chimpanzees must know that tokens signify 
potential food Very often this step is couched m terms of the reorgamza 
tion of the perceptual field (Adams, 1931, Lewin, 1942) The second step 
in explaining a learned act requires a statement of the manner in which 
the aroused expectation or signification leads to appropriate conduct 
It was originally assumed by Tolman that if a rat knew the behavior 
route to a desired goal it would follow the route He was criticized by 
Guthrie (1935, p 172) for not coming to grips with the problem of how 
the rat is led from its expectations to appropriate action The expectancy 
theorists are not unaware of this problem, and, faced with such criticism, 
Tolman (1955) has developed the performance aspect of his theory m 
more detail Specifically, he has proposed that the performance level of 
the learner varies with the value of a performance \ector, an intervening 
variable which depends upon the organism’s needs, upon the incentnes 
available and upon the amount of effort anticipated in acquiring a posi 
tive reward or escaping from punishment 

As is true of most theories, Tolman’s expectancy position does not 
propose to predict the details of conduct An expectancy can mediate 
the most varied behavior, as long as that behavior is consonant with 
anticipated consequences A rat expecting punishment at the end of the 
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maze may refuse to enter, may climb over the wall, or may simply crouch 
in the path and shudder. According to expectancy theory, it shows in its 
behavior that it anticipates an unpleasant consequence of running 
through the maze, but the details of the behavior cannot be accounted 
for by the mere acquisition of an expectation. Widely different perform- 
ances may serve the purpose of keeping the animal away from the place 
where it expects to be punished. 


DIFFERENCES AMONG THEORIES AND PROBLEMS 
RAISED 


The foregoing descriptions of the classical opinions of the nature of 
reinforcement will suffice to illustrate the main differences among them. 
To summarize: (1) The contiguity view holds that reinforcement consists 
in the strengthening of an S-R connection through the contiguous occur- 
rence of stimulus and response. (2) The effect theory proposes that S-R 
connections are strengthened as a result of subsequent satisfaction. (3) 
The expectancy position maintains that expectancies or cognitions are 
stiengthened through the contiguous experience by the organism of signs 
and signified objects, both stimulus events. Now obviously each of these 
statements contains two basic elements : one refers to the process which 
a remforcer hypothetically strengthens; the other refers to the nature of 
e reinforcer itself. The two components identify the main foci of theo- 
retical disagreement about the nature of reinforcement. One subproblem 
concerns the nature of the process strengthened; another involves the 
nature of the mechanism which does the strengthening 
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FIGURE 8 2 Double classification of theories in terms of (a) what js learned 
and (6) position on the law of effect Theories in the upper part of each box 
are single process theories Multiprocess theories are at the bottom 


1 Is learning a matter of establishing S R connections or of building S S 
cognitions ? This has been termed the problem of what is learned 

2 la reinforcement a matter of mere response elicitation, or is soma additional 
consequence of behavior necessary? This is the effect-contiguity (reinforce- 
ment nonremforcement) issue 

3 Do the answers to the preceding two questions apply generally or onl ) to cer 
tom forms of learning? This defines the single process-multi process issue 

The first two questions above are the ones which were the subject of 
controversy in early disputes over the nature of reinforcement The third 
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became more important in the light of accumulated evidence on the 
specific mechanisms of reward. Accordingly, experiments related to ques- 
tions (1) and (2) are reviewed in this chapter. Material relevant to 
question (3) is covered in the next chapter. 


HABIT OR EXPECTANCY? 

In the disputes which have arisen over the nature of the associations 
formed in learning and the role of reinforcement, the concepts of habit 
and expectancy have played a central role. It will be well to begin our 
discussion with a consideration of the meanings of these terms. Hull 
defines his concept, habit, as follows : 

Whenever an effector activity (r > H) and a receptor activity (S > s) 

occur in close temporal contiguity («Ch), and this >Cr is closely associated with 

the diminution of a need . . . there will result an increment to a tendency (A bBr) 

for that afferent impulse on later occasions to evoke that reaction. The increments 

from successive reinforcements summate in a manner which yields a combined 

habit strength (»H.) which is a simple positive growth function of the number 

of reinforcements (N). The upper limit (m) of this curve of learning is the 

product of . . . the magnitude of need reduction ... the delay (t) in reinforce- 

me ^i., and • • • the de £ ree of asynchronism (t') of S and B. . . . (Hull, 1943, 
p. -Li o ) 


A definition of expectancy at exactly this level does not appear in 
Tolrnan s writmgs, although it is clear (Tolman, 1938) that many of 

th! Tarn? if controm “e «« strength of an expectancy are 

grow T ™ , P ; stu ’ atcd by to govern the process of habit 

Slf * m n " “™ ber ° f pra0tice “sessions, the amount of 
Th f ” ™“s’ “f , tbe physicaI ebaracteristics of the learning situation, 
very really 

MacCorauodale nrir i esseD -tial similarity of the concepts, 

theory which inclm? pr0posed a formalization of Tolman ’s 

to Hull’s definition^ habit ni Whh° f s eX r P f CtanCy 7^ ^ r ° UgUy Pai ' allel 

cate, and R to a response cl Zf tL , 7^ t0 * ^ S * t0 * signifi ‘ 

’ hey define an expectancy as follows: 

The occurrence of the sequence Sx > R . « , tVl 

m close temporal contiguity) results iu ( f e adjacent members being 

pectancy (SiRS=). The strength im-r increment m the strength of an ex- 

of occurences of the sequence Th eaS6S HS a deee Lrated function of the number 

haIifi:“tlutXacfanr ment “ S **■**» - at a 

Xpectancy eme rge as (1) intervening variables 
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which (2) depend in the same way upon the number of learning occasions 
and (3) vary with certain aspects of the reinforcing stimulus (S 2 ). 
Moreover, (4) there is no description of habit as an S-R connection or of 
expectancy as a mentalistic concept, nor (5) is there any specified link 
in these definitions of the concept to its supposed behavioral consequences. 
But there are differences: (1) The definition of expectancy contains no 
mention of anything analogous to the law of effect, whereas Hull refers 
to it directly, holding that the diminution of a need is essential to leara- 
wg. (2) There are also trivial differences in the exact variables specified 
as responsible for the growth of learning and as to the locus (rate or 
limit) of their operation. 

From this analysis, it is clear that the only important difference be- 
tween the two types of construct, as presently formulated, is in terms 
of their positions on the law of effect. The other difference (S-S versus 
S-R) which has been so important in the history of controversy between 
habit and expectancy theorists takes on diminished importance when it 
becomes clear that these two basic concepts are, after all, concepts. It is 
on this basis that Kendler has labeled the problem of what is learned 
(S-S versus S-R connections) a pseudoproblem and a theoretical blind 
alley (Readier, 1950, 1952). 1 In the absence of a clear conception of the 
nature of scientific concepts, supposedly critical lines of experimentation 
were initiated in an attempt to decide once and for all which view was 
correct. In the main, the attempt was by the S-S theorists to discredit 
the S-R position, rather than vice versa. Strangely, the effect of the 
numerous experiments carried out in this tradition has been to strengthen 
rather than to weaken S-R theorizing, in spite of the fact that at least half 
of the studies have produced results which were negative to the S-R posi- 
tion. This is because the S-R theorists had to formulate more sophisticated 
theories which could explain the negative evidence. Such formulations 
have now eliminated many of the differences which once distinguished 
the two rival theories. Such concepts as mediation, r G , and incentive 
motivation, presently stressed by the S-R theorists, have a distinctly 
cognitive ring. The effect on the S-S theories, on the other hand, was 
to weaken them to the point where Kendler (1959) was forced to write: 
“Cognitive learning theory of the sort espoused by Telman ... is either 
sleeping or dying, [in part because] the negative attitude toward S-R 
psychology dominated their activities to such an extent that they ignored 
the development of their own theory.” (Kendler, 1959, p. 73). 

The most common forms of experimental attack upon the main issues 
separating the major theories were the following: (1) attempts to condi- 
tion subjects without employing one of the usual remforcers at any point 
in the experiment, (2) sensory preconditioning, (3) perceptual learning, 

(4) place learning, (5) nonresponse conditioning, and (6) latent learn- 
ing. It should be recognized at the outset that most of these experiments 

1 But set) Hozeboorn (193S). 
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have been conceived with a double purpose in mind. The first has been 
to evaluate the S-S as opposed to S-R hypotheses. The second has been 
to investigate the validity of the law of effect. 


Conditioning without Reward 

The ideal experiment for distinguishing between effect and con- 
tiguity theory would be one in which the response was simply elicited a 
number of times in the presence of some stimulus, to determine whether 
learning occurred in the absence of reward. Contiguity theory would pre- 
dict that learning would occur. Effect theory would predict that it would 
not. Although such experiments are simple and straightforward in concep- 
tion, not many of them have been done, because of the difficulties involved 
in evoking a response in a way which certainly involves no reward. There 
‘ have, however, been a few.VLoucks (1935), for example, attempted to 
condition the leg flexion of a dog to a buzzer when the response was 
elicited by electrical stimulation of the dog ’s motor cortex. He was unable 
to do so. When food was given following leg flexion, conditioning did 
oeeur. Along the same lines, Loueks and Gantt (1938) found that flexion 
elicited by electrical stimulation could be conditioned to a buzzer if the 
shock evoking the response were applied to the posterior root ganglion 
and stimulated pain fibers, but that it could not be conditioned if elicited 
in a way which involved only kinesthesis. It could thus be argued that 
the pain associated with the response was essential to learning, either as 
a part of the elicitation process or because its subsequent cessation was 
. 1 J e - redu cmg. For this reason, these experiments favor an interpretation 
m terms of the law of effect (Spence, 1950). 

Studies providing evidence in partial support of a contiguity interpre- 
ho?r,wI een COnee ;y ed ™ tMn the corL text of Guthrie’s theory which 
kaftot m ?l tati0U 0f a Iesp0nse a Si™ situation will 
(Seward “ 4,16 last one to — m that situation 

hat rats' llzff “ d =" t0n ' 1M6 >' Sa ™ 4 , example, showed 
out of tt ,T . the b “ “ a Sldm ® box ripply to be lifted 

With food 1? ti0n . gh . they 414 n,jt learn as well as rats rewarded 

reinforcing A mnr P off a PP araPus was not secondarily 

th tht„rSr e e 7 Way ° f taB ^ tke out of the situ- 

a.e.y r =T ht to render ;t “ 

using electroshock through rats’’ heads ' Performed such an experiment 
sciousness. When the shock wo f. to P ro4ll ee the state of uncon- 

Of an escape reaction tk ' &Pplled ““mediately after the occurrence 
that attempts to eondlt , t not It, thus, turns out 

duced results in favor of the law* ofTffeeT r6,Vard haT6 m ° St 044611 pr °' 
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Sensory Preconditioning 

In their reformulation of Tolman's cognitive theory of learning, Mao 
Cornuodale and Meehl referred to the concept expectancy as depending 

upon a sequence of events (Si » R » Sc) ^ mduiei a 

response Earlier expectancy formulae had either ignored or belittled the 
importance of the occurrence of a response, partly because it ™ 
that learning could occur without it One line of evidence favonng tins 
conclusion was the existence of a phenomenon called sensor!, precondl 
fuming in which an association apparently was established by the mere 
simultaneous occurrence of two sensory events, whether or not a response 

0C Th?basic paradigm for the sensory preconditioning expenment jn 
— of trials, (2, a — £* 

say, Si, (3) the ot er s mm -» ^ J{ Jt does tlie implication is 

the response tautei ‘ ‘ “ 7 the tet (preconditioning) phase of 

that some association w^forn d he transfer of the response from one 
the experiment which to* J ‘ hen lt oeetlrs , , s termed sensory pro 
stimulus to the other This e ' tbe first clear demonstra 

conditioning Brogden (1939c) is ereoiwu 

tion of the phenomenon and for t e name Kelly, 1934, Cason, 

Early attempt* ( ^ k ) °J T “ duce ..senary conditioning” as it was 
1936, Bogoslavski 1937) t0 P r0Q . m)f aegre es of success 

called prior to Brogden's study, ^'‘^“Uondusiie because 
By and large however effects of stimulus 

of a failure to employ contro ■ & P ^ stimu]us (Seidel, 19o9) 
generalization and famihan y ( los generalization comes about 

The necessity for the control lor ^ ^ t0 be 
because a response conditional P are m anotlier modality 

elicitable by other stimuli even if ioned t0 tbe light might be 

In the example above, the *“«» than because of any association 

given to the tone for this reas during the precondition 

which had been formed f b "n JU as cross modal 

mg Phase of the experiment Th P ^ precondl tiomng expen 
generalization, could easily o re sults on an artifactual basis 

ment and produce to employ a control 

In order to control for this paired The evidence for sensory 

group in -which Si and S 2 ha\e no e response to S 2 on the part 

preconditioning then, m m any Espenments *h,cb 

of the group for which the stim t mate some provision for 

contain the control for stimulus familiarity m 
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equating experience with Si and So, such as presenting them to the con- 
trol group as often as to the experimental group, but unpaired. 

In Brogden’s classical study of sensory preconditioning eight dogs 
received 200 pairings of a buzzer and light presented simultaneously. 
Then a flexion response to shock was conditioned to one of these stimuli, 
the hell for half the dogs and the light for the other half, to a criterion 
of 20 CR ’s in 20 trials. Finally the animals were tested in extinction with 
the other stimulus. A control group of eight dogs were treated in exactly 
the same way, except that the preconditioning sessions were omitted. The 
animals in the control group produced an aggregate of only 4 conditioned 
responses to the test stimulus. Those in the two experimental groups pro- 
duced /8. Clearly the preconditioning procedure had had an effect which 
somehow led to the transfer of the CR from the CS to the novel test 
stimulus. 


Other studies have shown that sensory preconditioning is not the result 
of differential familiarity with the test CS, as might have been the case 
m Brogden’s experiment (Bahriek, 1952 ; Brogden, 1947; Chernikoff and 
Brogden, 1949 ; Seidel, 1958). It has also been demonstrated that sensory 
preconditioning can be obtained when the conditioning procedure is 
instrumental rather than classical (Bahriek, 1952; Ram, 1947; Seidel, 
19o8) and with voluntary responses (Brogden, 1947; Chernikoff and 
rogden, 1949). When the appropriate controls have been employed and 
w en the importance of stimulus generalization has been adequately 
.a en ™ t0 aec ° unt > the magnitude of the sensory preconditioning effect 
. m, ^ Sma and su hjeet to extreme modification by instructional fac- 
e unimpressive quantitative dimensions of the phenomenon should 

faep be ° Wed t0 d f traCt fr ° m its theoretical importance. At 

terms ol 1!!? Preconditioning seems more easily interpretable in 
and Bitter™ a tian “■ terms o£ S ' R connections (Birch 

persuasion ^ P ° SeS & real P robl em for theorists of the latter 

*ts W reHed^p*; a" ^ the S-B theo- 
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cognitive interpretation. Inbriel Seyhold^m * 
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responses (r G ) have nmnninn +- never measured, (2) that these 
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elieitation of the response in -nl ° stage o£ the experiment and the 
of this explanation appear in TT e Dia grammatie representations 
Kubala, 1953; terman, Seed, and 

> , 19o9). A somewhat mo difi ed 
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version appears in Figure 8 3. This explanation, following Osgood (1953, 
p. 46011.), is often spoken of as invoking a process of mediation and of 
being closely related to mediated stimulus generalization (pp. 355 to 358). 

STAGE I - PRECONDITIONING 

s l{tone) ^ 

\ 

\ 

\ 

\ 

, \ 

211'Shll r (li 9 htJ s (lighl) 


STAGE 2-CONDITIONING 


S 2(l.glil!^. 

UOS {!hockr 


flight) *" s (li 9 H)^ 



STAGE 3 -TEST 

® I (lone) — r ( light} — ' s (ltghl) — ^ (flexion) 

FIGURE 8.3. S-R explanation of sensory preconditioning. Solid lines are un- 
conditioned connections. Dashed lines are conditioned connections. In Stage 1 
an unmeasured response to light (with its proprioceptive stimulus) is condi- 
tioned to a tone. In Stage 2, both light and its response produced stimulus 
are conditioned stimuli for ilexion elicited by shock. In Stage 3, the proprio- 
ceptive stimulus (evoked when the tone elicits its previously conditioned 
response to light) now elicits the flexion response. (After Bitterman, Reed, 
and Kubala, 1953). 

It is to be noted that this explanation of the sensory preconditioning 
phenomenon leads the S-R theorist into a position which is somewhat 
out of character, in that he has been forced to abandon the simple con- 
cepts of environmental stimuli and overt responses. He had found it neces- 
sary to posit the existence of covert, difficult-fo-recognize, responses ami 
to rely in Lis explanation upon the sensory consequences of these re- 
sponses In this shifting of ground, the S-R theorist mo\es, in effect, a 
step closer to expectancy theory in that the specification of the response 
is now in terras of implicit behavior. On the other hand, the mediation 
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hypothesis appears to be worth everything it costs the S-R position in 
terms of theoretical concessions, because it suggests an explanation for 
some of the previous negative results obtained in preconditioning studies 
and provides a rationale for experiments which should clarify the issue. 
If the first phase of such studies is, indeed, an ordinary classical condi- 
tioning procedure, the variables important for classical conditioning 
should be important for it. Of these variables, the two which appear to 
be most worthy of examination are the temporal spacing of the two stimuli 
(interstimulus interval) and the relative intensities of the two stimuli. 

In several of the sensory preconditioning studies the stimuli in Stage I 
have been presented simultaneously ; in others they have been separated 
by two seconds or more. Neither of these conditions is very favorable to 
the formation of a classical CR, and this may account for the failure of 
some of these experiments to obtain positive results. In much the same 
vein, the best arrangement for classical conditioning seems to be one in 
which the CS (Si, in sensory preconditioning) is relatively weak by 
comparison to the UCS (S 2 ) ; this suggests that such an arrangement 
might produce stronger preconditioning than has usually been obtained. 
Evidence from investigations manipulating these variables would go far 
toward substantiating or refuting the mediation hypothesis. Unfortu- 
nately there are very few experiments which are directly relevant. 

fo^rl tte °i simultaneous, 

. CS " _ UCS tnterval l.o seconds), and backward (“UCS” 
- CS interval l.o seconds) presentation of the stimuli in the precondi- 

StaSi P a„7 experiment . light and buzzer as the two 

stimuli thev , aS jeetS ' Af,er 3000 presentations of the 
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ditioned responses occurred in a series of 10 trials Finally there was a 
test consisting of 100 reinforced trials with the other stimulus In this 
test the forward conditioning group was superior to the simultaneous 
and backward conditioning groups, which were, themsehes, indistin- 
guishable All of tile preconditioning groups were superior, however, to 
three appropriate control groups The mam support for the mediation 
hypothesis is m the superiority of the preconditioning group originally 
trained with the stimuli presented m a forward order 
Certain of W J Coppock 's (1958) results, m a study using human 
subjects and the GSR, may be interpreted in a similar way Coppock 
ran 5 groups in the experiment outlined in Table 14 Note that, in addi- 
tion to running the usual control and conditioning groups, he ran two 
pre extinction groups for the purpose of extinguishing the associations 
established during preconditioning and an inverted preconditioning 
group for which the procedure corresponds to Silver and Meyer’s back- 
ward conditioning one The results of the Coppock experiment appear in 
Figure 8 4 The most straightforward statement of the outcome is that 
the three preconditioning groups were all superior to the control and 
inverted (backward) conditioning groups By the usual statistical stand- 



FIGURE 8 4 Results of W J Coppoc k’s experiment on sensory precondition 
ing The symbols are to be read SPE = standard pre extinction, IPE — 
inverted pre extinction, C •=* control , PC = (standard) preconditioning, 
IPC = inverted preconditioning Note that groups C and IPC are inferior 
to the other three groups The inferiority of group IPC supports the hypothe 
sis that for sensory preconditioning to occur, the order of stimuli during the 
preconditioning phase of the experiment must he in the forn ard-condit loiiing 
sequence Graph from W. J Coppock (1938). 
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ards there was no significant difference between the latter two groups 
individually or among the first thi’ee. In general these results, then, 
support the mediation hypothesis. They do, however, raise a problem as 
to why the pre-extinction procedure failed to accomplish its purpose. 

Finally there is the parametric study of Hoffeld, Thompson, and 
Brogden (1958) which, although it was not conceived within the medi- 
ation hypothesis framework, is directly relevant. These investigators 
trained five experimental groups of eats in a preconditioning arrange- 
ment with tone and light as the first and second stimuli, separated by 
0, 0.5, 1.2, 2.0, or 4.0 seconds for the different groups. There was also a 
control group which was placed in the apparatus for the same amount 
of time hut received neither of the two stimuli. All of the cats were 
then trained to avoid a shock in a wheel-turning apparatus, with light 
as the CS, to a criterion of 90 per cent CR. Tests for sensory precondi- 
tioning followed, measured as number of responses to extinction. In these 
tests, the control group gave no CR’s, but all of the experimental animals 
gave at least one. Median numbers of responses to extinction were : for 


the 0-, 0.5-, 1.2-, 2.0-, and 4.0-seeond groups, respectively, 2.5, 3.5, 3.0, 
5.5, and 8.5 responses. This suggests that the function relating the amount 
of sensory preconditioning to the interstimulus interval is still increasing 
at 4.0 seconds, a curve which is very different from that obtained in 
studies of ordinary conditioning. This result calls into question the theory 
which demands that the optimal interval occur at about 0.5 second. 

Other studies of the mediation hypothesis have attempted to manipu- 
ate the mediating process in some way. Seidel (1958) obtained sensory 
preconditioning m an experiment with rats and, by shifting drives, 
s lowed that the mediators are not responses specifically associated with 
! ’ m;t Y Iekens and Eri SSs (1951) presented two stimuli, 
had E ,reoonditi °“nsr Phase of their experiment and 
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Mean Number or Responses on Test Trials svith Shock Omitted 
(Wickens and. Briggs, 1951) 

Preconditioning Group ’ ' jj” 

I “Now” to tone and light together ~ ' *" " ’ 

II “Now” to tone and light separately 7 2 

HI “Now” to light only 21 

IV “Now” to tone only 0 2 


Perceptual Learning 

In discussing perceptual learning it will be well to beep two different 
meanings of the phase separate (Gibson and Gibson, 1955) On the one 
band, perceptual learning may refer to the proposition that learning, 
or at least some learning is a process closely related to perception and 
consists m a reorganization or integration of perceptual processes S S 
learning theory advocates such an interpretation At other times, the 
expression perceptual learning refers to the fact that perception itself 
depends upon experience It is this second sense of perceptual learning 
to which we turn briefly in this section 

There is evidence in considerable variety to indicate that a part of 
what we call perception is learned It has been shown that (1) practice 
or familiarity increases the speed with which materials can be recognized 
tachistoscopically and this effect is upon speed of seeing and not upon 
the mere sensitization of a restricted range of motor responses (Neisser, 
1954) , (2) perceptions of the simplest properties of the risual world, 
both in animals and men are acquired, and if experience with this 
environment is prevented there may be a serious impairment of visual 
capacity (Senden, 1932, Riesen 1947), (3) age improves size shape, 
and brightness constancy (Brunswick 1956) , and (4) exposure to visual 
stimuli makes the organism more capable of dealing with these stimuli 
in a subsequent learning situation 

Some of the evidence in this last category conies from the field of verbal 
learning (Hovland and Kurtz, 1952) and is beyond the scope of this 
book, some of the rest is related to discrimination learning and Will be 
treated later The work of E J Gibson and her associates may be cited 
at this point to demonstrate the validity of the concept of perceptual 
learning In the first of their studies these investigators (E J Gibson 
and Walk 1956) reared the rats in their experimental groups with dis 
tinctive stimuli, Hack circles and triangles, on the walls of their cages 
Most of the other distinctive stimulation in a normal environment was 
excluded by surrounding the cages with blank white cardboard walls 
The control group saw only the blank walls At the age of about 00 days 
these two groups of rats were taught to discriminate the circles from 
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the triangles in a typical discrimination procedure. In this procedure the 
animals were released from a starting box into a choice chamber which 
contained samples of the two stimuli, side by side, on the wall at the end 
opposite to that through which the animal entered. In order to prevent 
the subjects from learning a simple position preference, the stimuli were 
randomly shifted from side to side on various trials. The animal was 
rewarded with food if it nosed open a small door in the center of the 
correct stimulus, which was the triangle for some subjects and the circle 
for others. Learning curves for the experimental and control groups 
appear in Figure 8.5. Quite obviously previous exposure to the stimuli 
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it is difficult to see how o\ert, molar responses, even if they were condi 
tioned to the stimuli, could be of much help in mastering the discrimina- 
tion problem Such responses transferred to the learning situation would 
be of no particular use, because the stimuli were shifted from side to side 
Moreover, for half the animals, the circle was positive, and for the other 
half, the triangle was positive Thus, to maintain the S R position, it is 
necessary to say that some subtler form of behavior such as, perhaps, 
Wyckoff’s (1952) observing responses were strengthened during the first 
part of the experiment, thus the experimental animals would be better 
prepared to "pay attention" to the relevant cues than the animals in 
the control group Another S R explanation might hold that perceptions 
have response properties and, like other responses, can be modified by 
learning 


Place Learning 

In 1946 there appeared the first of a series of studies (Tolman, Ritchie, 
and D Kahsh, 1946a, 1946b, 1947a, 1947b, Ritchie, 1947) 3 which argued 
m favor of place learning, rather than response learning, as character- 
istic of the behavior of rats in a spatial maze The second of these studies, 
for example, opposed the two kinds of learning in an attempt to provide 
a definitive comparison of their relative importance The apparatus 
employed was the maze shown in Figure 8 6, which differed from most 



FIGURE 8 6 Apparatus for the study of place and response learning S, and 
S 2 are starting points Fi and F 2 are the locations of goal boxes containing 
food In this experiment, rats began some trials at Si and others at S 2 Under 
conditions of place learning, food is always in the same place, say, Fj In 
conditions of response learning, food is shifted from Fi to F 2 in a way which 
requires the animal ah* ays to make the same response, say a right, turn in 
order to get food {Tolman, Ritchie, and Kalisb, 1946b). 

» See also Tolman and Qleitman (1949), Ritchie, Hay, and Hare (10J1), Ritchie, 
Aescblunan, and Fierce (1950) 
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mazes in having two starting points, Si and So. In the experiment the 
rats sometimes began a trial at one starting point, sometimes at the other. 

There were two groups of rats in the study. One group, a response- 
learning group, had to leam to make a right turn, no matter wdiich was 
the starting point. If they began at Si, they had to go to Fi to be fed; 
if they began at So, they had to go to F 2 . The other group -was a place- 
learning group, which was always reinforced in the same place, say Pi, 
without respect to the starting point. 

The latter, place-learning, group mastered the maze very much more 
rapidly than the response-learning group. Whereas the average rat in 
the response-learning group did not meet the criterion of 10 successive 


correct runs in 72 trials, the place-learning group met this criterion in 
only 2 trials. Obviously the place-learning conditions favored learning 
in a way the response-learning conditions did not. Again the simplest 
form of S-R theorizing fails to predict the outcome. 


More sophisticated versions of S-R theorizing, however, suffer no 
embarrassment from the place-learning experiments. The problem, it 
should be clear, is with the term place. To use it, as Tolman and his 
collaborators dearly saw, is to identify a region of the environment in 
terms of specific stimuli. According to Tolman, Richie, and Kalish, one 
o t e t m » s a ra t learns in a maze is “a disposition to orient towards 
the place where the food is located (under the window, to the left of the 
rathator, and so on)” (1946b, p. 221). And, in the demonstrations of 
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to response learning. The whole nrolh.m “™omnent, animals resort 

sensory control of the mace issue ^Hoitt 193sf wb'T^T'f ^ “‘f 
research, to the discovery that rats are ’ 1 « * Ied > after muob 

whatever sensory information is provided and ™ maidng " Se 

provided, and master a maze on this basis. 


Learning without Responding 

P ° Siti0ns theoretically might be 
as a result of mere a ?*** 18 P ° Ssible ^out responding, 

ation, or whether ^ 

UbC , ln some sense, occur. 
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Various attempts to eliminate tlie response from the learning situation 
ha\e ranged all tlie way from giving rats a ride through a maze m a 
little cart (McNamara, Long, and Wike, 1956) to pharmacological and 
surgical immobilization of the response mechanism during sensory stimu 
lation Studies in this latter category have most often used the strategy 
of conducting “conditioning” trials when the subjects were immobilized 
by an administration of curare or a curare like drug and then tested for 
the presence of a CR when the effects of the drug had worn off The 
results of early studies of this sort 4 were sometimes positive and some 
times negative It now seems that the negative results may have arisen 
from inadequacies in the training procedures, the drugs, or both More 
recent investigations usually have been successful 

Using cats as subjects. Beck and Doty (1957) immobilized the flexion 
response to shock m various ways and paired a 2 second shock with a 
tone to determine whether conditioning would occur For three cats 
immobilization was produced pharmacologically by injections of bulbo 
capnine, for three others the bulbocapmne injections were combined with 
crushing the motor nerves for the flexion response, and for two other 
cats, the immobilization was produced by bulbocapmne and midbram 
lesions Seven of these 8 animals showed substantial levels of conditioning 
when the effect of the drug had disappeared and the animals had recov 
ered the use of the surgically immobilized limb Light and Gantt (1936) 
and Kellogg, Scott Davis, and Wolf (1940) had previously shown that 
conditioning can occur when the response is prevented by crushing the 
motor nerves 

In a related experiment, Black (1958) showed that extinction occurs 
if the CS is presented alone while the response is immobilized Dogs were 
first trained to turn their head to one side, in response to a tone, to 
avoid shock Following this training control subjects received 50 non 
reinforced tone presentations in a normal, responsive state Experimental 
subjects were injected with d tubocuranne chloride, which produces a 
flaccid paralysis on the skeletal musculature, and given 50 extinction 
trials on which the response, of course, could not occur The median 
number of trials necessary for extinction (counting the 50 under curare) 
was 91 for the experimental group, the control group animals did not, 
on the average, extinguish m 450 extinction trials It should be noted 
that extinction is actually considerably faster under curare than in the 
normal mobile state Such experiments bring the problem facing the S U 
theorist into sharp focus Obviously the conception of learning as involv- 
ing an association between stimuli and responses, when the latter arc 
conceived in terms of gross molar events is untenable 
* Girden (1040, 10-1 So, 1942b, 1942c), Girden and Caller (1537), Hallow (1540), 
liar low and fccttlage (1935), Harlow aad Slag ner (1933) 



226 


CONDITIONING AND LEARNING 


Latent Learning 


The term latent learning, as it is used in experiments designed to test 
the relative merits of habit and expectancy interpretations of learning, 
refers to learning which occurs (a) without immediate consequences on 
performance and (b), according to the expectancy point of view, with- 
out reward. The argument is mainly about the second of these two propo- 
sitions. The expectancy position is that learning can occur without re- 
ward, and that mere experience in a situation is sufficient. The translation 
of such learning into performance requires, in addition, the existence of 
an incentive. Prom such a point of view, the trouble w T ith the typical 
learning situation is that these two processes are confounded in a way 
which makes it appear that the incentive is an essential condition of 
learning. The latent-learning experiment is a method for separating the 
two processes. 


The reference experiment in the field of latent learning is that of 
Tolman and Honzik (1930) in which rats ran a maze for 10 days with 
(control group) or without (one experimental and one control group) 
reward. On the eleventh day reward was introduced for the experimental 
group previously run without reward. The results of this experiment 
appear on Figure 8.7, where it is quite obvious: (1) that the rewarded 

etter an n ° n ' rewarded rats >' (2) that, even without 

vZZt l ^ T S T ™ pr0vement; and ( 3 ) that the introduction of 

crlase ffi n nn h T ^ "" 6Xperimeilt led to an immediate de- 
crease in number of errors in the group treated in this way This last 

‘ a : m °? T* Mnd ° £ evid ““ f ” -isteifce of latent 
about the tlT 'i'' 1 ' ™ tS . ™ thls grou P kr,ew considerably more 
, was ree^rTd , „ X T T™ 8 beha ™ r had Seated, and all that 
Se eX of ‘ eb ! an adequate reason (incentive). 

meLTa nerftrn ^ & “ th<! assumptionutfiat reinforce- 

1 “XraUy theTXe 6 T Iather tia “ controlling learning. 

refntXnyX^XbrX! the ° riste . did take this apparent 
number of questions to ask ■ the "tw aSSDai ? tK>n lyin S down. They had a 

1. Is the phenoTenon S',1 7 m ° St were the following: 

effect be obtained routinely! ExX™”? depcildable! That is, can the 
showed that it did not alwnv P erimen ts began to be reported which 
Such demonsfradon“e y r S ^f ar «~p!e Eeynolds, 1945c). 
Meehl (1951) reviewed 48 relevant stn rH 6 ' Ma eCorquodale and 

positive category. It seems “^d XX Tf? 3 ° * tbem ^ 
of the phenomenon can nn lmur ° 1 • concludes, that the reality 

studies yielding positive and ne^af 6 6med ' Table 16 summarizes the 
tion developed below. ° We results 111 terms of the elassifica- 

2. Has reward actually been eliminated from the situation! This qnes- 
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FIGURE 8 7. One of the classical demonstrations of latent learning For 
description of experiment, see text Data from Tolman and Honzik (1930). 


tion may be asked from a number of different angles For one thing, the 
initial improvement in performance in the Tolman and Honzik experi- 
ment can be made a source of embarrassment to expectancy theory If 
the sudden drop after trial 11 is to be attributed to reward, is it com- 
pletely logical to fail to interpret the previous, more gradual improve- 
ment in the same way? With this as an entering wedge, it is possible to 
suggest a number of sources of reward which might reinforce behavior 
on the first 10 trials These include secondary reinforcement through 
being handled by the experimenter or returned to the home cage, rein- 
forcement provided by escape from confining euls-de sac during the early 
trials, and reinforcement stemming from the satisfaction of curiosity or 
activity drives Such possibilities led to a multiplication of the number of 
experimental designs employed in the study of latent learning Mac- 
Corquodale and Meebl (1954) added one type of experiment to the list 
proposed by Thistlethwaite (1951a) and identified fit e varieties 

a Trials without reward followed by the introduction of a reward 
appropriate to the animal's motives This is the design employed m the 
Tolman and Honzilc experiment taken as a reference Similarly Kara and 
Porter (1946) ran rats to a food reward m a complex maze For a control 
group with no previous experience in the maze, the number of trials re- 
quired to learn it was 10 25 For an experimental group which had been 
given five unrewarded trials, the number was only *J 83 

b Free exploration followed by the introduction of reward Lashley as 
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TABLE 16 


Summary of Representative Latent Learning Experiments 
Classified in Terms of Type of Experiment and Outcome 


Type of Experiment Outcome 

Positive Negative 

1 Blodgett (1929), Elliott Heehl and MaeCorquodale 
Trials without reward (1929), Herb (1940), Muenz- (1951), [Reynolds (1945e) 
followed by introdue- inger, and Conrad (1953), 

tion of Teward. Simmons (1924), Tolman and 

Honzik (1930), "Wallace, 

Blackwell, and Jenkins (1941), 

Williams (1929), Meehl and 
MaeCorquodale (1953) 

2 Buxton (1940), Daub (1933), 

Free exploration fol- Haney (1931), Earn and 
lowed by introduction Porter (1946), Lashley (1918 
of reward. 


3 Kendler and Levine (1953), Kendler (1947a), Maltzman 

Trials satiated, goal MaeCorquodale and Meehl (1950) 

objects present. Tests (1951, Meehl and MaeCor- 

under relevant drive, quodale (1938), Seward, Levy, 

and Handlon (1950), Spence, 

Bergmann, and Lippitt (1950), 

Spenee and Lippitt (1940), 

Szymanski (1918), Thistle- 
thwaite (1951b) 


\ Bendig (1952), Christie 

Trials under irrele- (1952), Diesenroth and 
vant drwe followed Spenee (1941), Strange 
by shift to relevant (1950), Thistlethwaite 
nVe ' (1952b), Walker (1951), 

Walker, Knotter, and De- 
Valois (1950), Johnson 
(1952) 


5 Gilchrist (1952), Iwaharn 

xploration followed and Marx (1950), Seward 
by reinforcement in (1949), Strain (1953) Tol 
stmctive goal box, man and Gleitman (1949b) 


Christie (1951), Eeher 
(1951), Gleitman (1950), 
Grice (1948a), Kendler 
(1947b), Kendler and Kan- 
ner (1950), Littman (1950), 
Shaw and Waters (1950), 
Spence and Lippitt (1946), 
Walker (1948) 


Denny and Davis (1951), 
Levy (1952), Mintum (1954), 
Seward, Datel, and Levy 
(1952) 


a * es .P erim -t of this sort. Later Haney 
periods of free exploration & C ° mple f 14 ' unit maze and long 

for four 18-hour periods. A control^^ hlS a ? lmals . to live in the maze 
exploratory experience but in a • gr ° 1 Up reeeived similar handling and 

ly tests 

emwaxte (1951b) ran rats, satiated for 
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food and water, in a Y maze which contained a dish of wet mash in one 
arm Interspersed with these satiated trials, there were test tna s 
which the rats were 22 hours hungry and there was no food m the maze 
On these test days there was a stronger tendency to tahe the food side 
t° an on the satiation days indicating that under satiation the animals 

had learned the location of food , , ,, i 

d Trials under a strong irrelevant drive followed ly ajuft to the rele 

landing platform Then the rats ^ec^ ^ ^ ^ 

swam directly to 

hours of food deprivation Thirty nve oi 

the platform which had contained foo 

, Exploration followed, by reinforcement tn a esplore a T maze 
Seward (1949) allowed 32 ™ts, ve or sn^ ^ Qn the fourt h day he 
for 30 minutes a day o\er P always been at the 

fed the animals in a m an ordinary trial in 

end of one arm of the T T * p01nt m the maze and chose the 

whieh they began the run £ ra ts m the exper , m ent, 28 

path to one goal bon or th oth« just been fed 
went directly to the box wh ^lymg negative reinforce 

One of the few studie Strain (1953) first allowed rats to ex 

ment also belongs 111 th * s cat ^ J* sWin -mg doors into six perceptually 
plore a runway divided w JfJlr six days of this without reinforcement, 
distinct compartments o ° compartment Then he rein 

he shocked the nb « Tour different groups entered the 

troduced the rats to the sppw M t hey had in the period of 

runway at the four middle comp ^ ommaIs left the compart 

free exploration The qnes i d th test would they now tend 

ment into which they were m reduced 1 on ^ compartment Jn winch 
to leave in the direction le S ^ rats ln the experiment 53 did 
they had just been shocked f “ duced mt0 the compartment 

exactly that Moreover, s]l0 „ ed t h,s tendency to a greater 

adjacent to the shock comp uted controls showed that the 

degree than any of the 5 th ' ” ^ must have depended upon mem 
latent learning evidenced ‘ ments rather than upon extra maze cues 
ory for the sequence of compa L earmng The foregoing experiments 
Variables Controlling LaU s stre ssing the point that latent 

^Sfb^^rr^ to omit the facts « - 
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the latent-learning effect is frequently weak and transistory; and (2) 
that, in many experiments, it failed to appear at all. These considerations 
raise an obvious question: What are the conditions which favor the oc- 
currence of latent learning ? There has, as yet, been no definitive answer 
to this question. But there have been numerous suggestions as to impor- 
tant variables, most of which have been summarized by Thistlethwaite 
(1951). Some of the most important general classes of such variables are 
the following : 


1. The maze pattern. Some of the most impressive of the demonstra- 
tions of latent learning have been in complex mazes. These include many 
of the studies in the first and second designs outlined in the previous 
section. The significance of this fact is that it is difficult to construct a 
complex maze without making the majority of true-path sections goal 
pointing. Thus, for latent learning to occur, it would be necessary only 
for the animal to learn the general location of food on its first reinforced 
run, in terms of some extra-maze cue. In attempting to go directly to that 
place a decrease in the number of errors would be inevitable. This might 
be caHed a place-learning explanation. One of the points to recommend 
this interpretation is that it accounts for the fact that, so far as latent 
learning is concerned, complex mazes appear to be easier to master than 
simple ones An°th e r line of evidence in support of such an explanation 
comes from the results of Type 5 experiments in which rats are first al- 
J , ° ex P ore a maze and then fed in a distinctive goal box located 
T„.- la ”, at Its pre ™ ns “1 subsequent position in the 

r u A L r • lvard ’ 1949) ' This pr ° ce,iure ( wuch mi s w be 

Mu°re Of lawT “ s ratS “ t0 the locatio11 of l°«i) leads to a 

comix mf f’rLT t0 , aPP f r ' Theie are - btnvev er, data obtained in 
Blodgett (19901 ' C ^ place ‘ Iearnm S interpretation will not explain, 
exploited period T r f ™ P ° f ra * S iaciward thl '»ngh a maze in the 

£rs™:T2 a ,: h ' “■ for ae — d 

but one of the trim nail. earning m spite of the fact that all 

.atentTeHIal orl 0b ™^, 

play some part Goal +; r e ^th respect to intramaze cues must 
ILd Meriden C8 lor particularly important in this 

trained without reward ’ P°mt, Seward (1949) showed that rats 
fed in one of them also faffed ^ ^ 

period Jhe S ubject°durin g the training 

whether or not the phenomenon occurs' ° The^Tt “ determinmg 
simple one and no doubt varies effect> however, is not a 

periment to another and from n ^ f ° rm ° f tlle la tent-learning ex- 
position leads to the expectation' Sv? an ? ther - Tolman’s (1948) 

with latent learning espeeiallv XT ° motlvat a°n would interfere 
° especially m Type 4 experiments, where the subject 
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is highly motivated for one goal object but to show latent learning would 
have to discover the location of a reward appropriate to another drive 
The reason for this is that strong motivation, according to Telman na 
rows the rat’s cognitive map and prevents lus paying attention to objec 

other than those particularly relevant to his needs _ 

By and large the evidence is in support of this interpretation For one 
thing studms of latent learning of the Type 4 variety have been pr* 
domfnantly negative, whereas Type 3 studies (which i are “ 
Type 4 except that drive is low during the imt al tra.ningj have been 

who studied the ro e f d hol]rs ran a T maze , there was no water 

pnved of water for 0 6 1,! ^ a ol tram ing under these 

but food was present m one ar On the initial hunger trial the 

conditions, the rats were made hungry On the nntn ^ ^ ^ ^ ^ 
percentages of rats * st {0 hlghcst drive during training 

the four groups t0 be the most effective condition 

Thus training under low dn R1td w j ess direct evidence 

for the establishment of that incidental 

to the same point comes from P better under low drive 

learning ol which latent learning ^ 

than under high drive ( Br ^ * the role of motivation in latent learn 
A somewhat neglected aspe , drives a s must be done in a 

mg concerns the a^^Such changes have a variety of influences which 
Type 3 or Type 4 study Such e g o£ latent learning 

may tend to assist or mt ^ r “ 7 r ^ us ap p” ranC e of what seems to be 
Some may even lead to the J at a y P In thls last category there is 
latent learning but is no ICendler and his associates that am 

the phenomenon eapenment, not latent learning) 

mals trained (in an ordin ry 1 S switching the drive to thirst 



Altchek and Peters 1952, 

pleasant to a thirsty rat Thu y P o£ fte or dmanly used 

from hunger to thirst m a would result in the animals’ abandon 

in Type 4 studies of latent learms^ ^ The ^iteh would occur for 


Type 4 studies of latent learning^w™ ^ The switch would occur for 
mg the side previously reinf ° r ,, t (he annual know that water is on 
reasons which in no way “y account for the fact that 

the previously non «»%£££ „ rlg ,nal .raining is under food dc 
latent learning occurs bette tl0]J (Vmeberg 1953) 

privation than under water dep mfluence latent learning cs 

The effects of altering » »» “ hfe hls torj of the laboratory 

penments in another way Ordma . T satisfied at about the same 

rat hunger and thirst occur together ood a ^ 1M3 , Bull 

time Tins probably accounts in part 
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1933) that rats sometimes have very great difficulty in discriminating 
their drive states. If this difficulty carried over to the Type 4 latent- 
learning experiment, the animal might not recognize the change from 
hunger to thirst or vice versa as a change at all, and for this reason, might 
continue to respond with the originally reinforced response. Operating on 
this assumption, Christie (1951) gave animals training in discriminating 
drives and then carried out a Type 4 latent-learning experiment to deter- 
mine whether animals with such experience might not show more latent 
learning than rats without such a history. They did not, and Christie was 
led temporarily to reject the hypothesis that drive discrimination is an 
important factor in the latent-learning experiment. 

3. Previous experience. A more general conception suggested by 
Christie s experiment is that the history of the organism may determine 
whether or not it is capable of displaying latent learning. Christie’s 
earlier negative result, then, might only indicate that a history of de- 
privation is probably not the critical thing. Other possibilities are that 
the most effective kind of prior experience is that which creates a general- 
ized exploratory driie, leads the animal to notice objects in the environ- 
ment, diminishes the amount of fear aroused in a novel situation, or pro- 
vides an irrelevant incentive with unusually powerful secondary rein- 
oreement value. There is presently no very good reason to choose one of 
these interpretations over any other, but there are experiments which 
s row, beyond much doubt, that previous experience does influence the 
outcome of latent-learning experiments. Christie (1952) repeated his 
earlier experiment adding some additional groups which had had ex- 
ploratory experience in a non-maze situation prior to the latent-learning 
imnre^h the exploratory training contributed more 

history 5 of de n S abUity to ^splay latent learning than the 

SLffiation^ Srf ° n ^ d ’ altll0Ugl1 111 tkis e -p™nt the drive-dis- 

to , iearning ’ to °- christie ’ s resuits ais ° 
surest that exploration under satiation is, if anythin- superior to ex- 
ploration under hi-h drive \ i n +„, - Cl. ! °» su Perior to ex 

SnrrKS" "f ** ™ uZy motivated 
satiated. rners than those who explored while 

orST fonSvedt * ^tent learning was 

ward and, therefore as a chalWr ** d< - monst . rate teaming without re- 
Reinforcement theorists accented tv °- ^ Validi . ty of the law of effect, 
and the battle was ioined Sin +i interpretation of the phenomenon, 

articles on latent learning, reporting SCOl ’ eS ° f published 

ments conducted bv some nf t. even greater number of esperi- 
ted by some of psychology’s finest craftsmen in the art of 
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experimental design In spite of the effort and resourcefulness devoted 
to the problem, lion ever, one is forced to conclude that experiments on 
latent learning have contributed little or nothing directly to the issue they 
were designed to settle It may be instructive to outline some of the 

reasons for this , , , •, j 

The basic reason, no doubt, is that the situations and behaviors involved 
in latent-learning studies are so complex that it is never posaMe : to state 
with complete certainty that a given experiment hits a ico mplrshed its 
primary objective, that of temporarily eliminating reinforcement for one 
£oup If subjects m the experiment 

complete absence of mottvated This 

a rat ran the maze is to y * f uncontr olled rewards for such 

raises the possibilities of the " 1954 Deslde rato, 1956), curiosity 

drives as activity (Kagan and Berkun, , restriction avoid 

(Butler, 1953), » 

ance (MacCorquodale and ’ lblIltie s (1) that there is incomplete 

relevant drives, studies and (2) that reinforcement is 

P was present in some form m any 

experiment on latent learning . . £ tbe existence of some sort 

0,1 the other hand, a mere the results of 

of reward does not pr°' rlde ^ stlll rcqulred Is some way to trans 
latent learning experimen In ^ re „ aI . d tbe problem of the S R 

late this learning into perf of tbe e j pec tancy theorist Hull’s 

theorist becomes the same (training satiated, tests 

(1952) handling of the results^ ejpenment » typical 

under relevant drive) fo He begins with the assumption that 

of a number of such rein£ orc,ng even to a fully satiated 

the sight of food l or water ^ ^ througb the evocation of an 

organism, and that t h assumes that the r 0 ■* So 

ro * So mechanism Moreover > „ afferent from 

mechanism evoked by the s ight To ,Uustrate the application of 

that elicited by water (r« '» stg jn tbe presence of the cues 

these ideas consider the m „here a left turn (R l) leads to foot! 

tS c ) at the choice point in a ^ orll , ca l to this argument that 

and a right turn (Rn) Ieads , t0 th<! fi na l overt response, oilier 

the choice point cues ehcl , m „ 1|lcll scrae the function of provld 

responses of looking bac , imu i a tioa They are, in Hull’s carliercr- 
mg discrete proprioceptive m the Jms of Muenzmger (19i«, 

mmology, pure stimulus act* , or “ , be * looking to-tl.e right 

vicarious trial and error (ViJvj 
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and looking-to-the-left responses as r K and r L we may begin tbe construc- 
tion of a diagrammatic explanation of latent learning : 

Tp > Sl > Tp > Sp > Rp 

ScC" 

s 

Tr > Sr > rvr > Sw > Rr 


In brief this diagram says that the implicit behaviors elicited by S c have 
distinctly different proprioceptive consequences (sr and Sl) which, be- 
cause they previously have been followed by food and water, now are 
capable of eliciting the appropriate fractional goal respo ns es (rp and r\y). 
This is possible in the absence of, and in anticipation of, the molar re- 
sponses Rr and R K , because the molecular, implicit, internal, nature of 
rp and rw permits them to occur covertly. Given this ass um ption, it is 
now possible to assume the important link in the conditioning process for 

the explanation of latent learning : that is s F » R L and s w » 

Rr. 

. If ’ now ’ the animal is made thirsty, the situation changes in the follow- 
ing way: The thirst drive and its associated drive stimuli S T , in the past, 
have been conditioned to r w , so that now : 


St 




Sw 


Rr, 


and the animal goes to water in the T-maze. 

flTl+kr 6 r!! n f D i S ab0Ut tMs ex P lana tion which need to be made : 
Other nrnha'w Vf? & / P6 3 vers j on latent-learning experiment. 

Obviouslv if 7 1 eXplanations are required for other types. (2) 

just presented e5planatton evea in the oversimplified form 

clear that if behavior Tere to' be °exnlaR ^ Psychology. It seems quite 
tions would >,Q-n- a p t ^plained simply, the simple explana- 
W wnrid have been dtscovered long ago. (3) The mechanism, 

concept itself A <5 * Sf ’ ^ a ^ out as ex Pectative as the expectancy 

tra^cC tea thTs 7 w m ‘ imeS brf0re - “ •* 

esplanations which attempt to hr’ , r ™ u ' n ll - v leen forced to develop 
S-R framework Such da 1 co " ni ti v e phenomena within an 

sharp diflZces w“e“ 

inforcement. There are in fact p ns tle elasslc al theories of re- 

positions of the 1930 ’s and nrobahl 617 ™ Proponents remaining of the 
be settled by a single perfidy oon^ “T Wh ° beUeTe that the 
are not that exactly formulated InT' r eIpe ™ ent - T be theories simply 
in the field of learf oTeem‘ to l J-T ° f tUs ™P°ssibility, interest 

ally oriented set of problems. to neTchapto. 
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SUMMARY 

In the 1930’s research in the psychology of learning -was dominated by 
three major systematic points of view (1) Contiguity theory which held 
that the conditions necessary for learning were simply the occurrence of 
a response in the presence of a stimulus Later on that stimulus theoreti 
cally, would evoke the response No additional principle of reinforcement 
was hypothesized (2) Effect theory which maintained that learning re 
quires (a) stimulus response contiguity and (6) reward m the form of 
satisfaction or drive reduction (3) Expectancy theory according to 
which learning consists in the acquisition of knowledge and involves inter 
sensory (S S) associations, rather than the S It connections hypothesized 
by the other two major positions Expectancy theory was also typically 
a non reward position 

The issues which divide these major theories are essentially two (1) Is 
reward (effect) necessary for learning, or is mere contiguity (whether 
S S or S R) enough ? (2) What is learned — stimulus response connections, 
or knowledge in the form of S S associations ? These two questions, to 
gether with the alternative answers to each, gives rise to four possible 
general theories of learning (1) S R contiguity theory, (2) S R effect 
theory, (3) S S contiguity theory and (4) S S effect theory No one has 
ever proposed the last of these for more than certain limited forms of 
learning As this last statement implies, several theorists have maintained 
one position for some kinds of learning and another position for others 
These multi process theories will be described in the next chapter 

Experiments designed to settle the issues implicit in these various sys 
tematic positions have most often consisted of attempts on the part of the 
S S theorists and non reinforcement theorists to discredit the S R effect 
position More specifically these efforts have involved experiments of six 
different kinds (1) attempts to condition subjects without reinforce 
meat (2) sensory preconditioning (3) perceptual learning (4) place 
learning (5) nonresponse conditioning, and (6) latent learning Work 
on each may be summarized as follows 

1 Conditioning without Reinforcement There have been a few at 
tempts to produce conditioning without the use of a tj pical remforcer at 
any point m the conditioning procedure but they have been unsuccessful 
In cases where conditioning has occurred it has always been possible to 
demonstrate the presence of some remforcer which the procedure has 
failed to eliminate The reason there have been so few experiments of this 
sort is that it is very difficult to maintain behavior without tho use of 
reinforcement 

2 Sensory Preconditioning Thu sensory preconditioning espcrirocnt 
requires three phases winch intoho (o) the pairing of ti\o neutral stim 
all, (4) the conditioning of some response to one of these stimuli, and 
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(c) a test with the other of the two neutral stimuli. A successful demon- 
stration of sensory preconditioning consists in showing that the response 
conditioned to one stimulus automatically transfers to the other, as if 
during the first phase of the experiment an equivalence had been formed 
between these two stimuli. It is to be noted especially that, if such an 
association has been formed, (a) it seems to he of a stimulus-stimulus 
variety, and ( b ) the presence of a reinforeer, in the sense the effect theor- 
ists use the term, is difficult to demonstrate. 

There is no doubt that sensory preconditioning occurs, although, with 
adequate controls for cross-modal stimulus generalization and familiarity 
■with the two stimuli, the magnitude of the phenomenon may be small. 
Attempts to explain sensory preconditioning by S-R effect theorists 
have concentrated more on the S-R aspect of the problem than they have 
on the difficulties it raises for the law of effect. These explanations have 
adopted a mediational approach and have taken the position that the 
original phase of the experiment is actually a true conditioning pro- 
cedure in which undetected cue-producing responses (r G > s G ) 

elicited by one stimulus are conditioned to the other. During the second 
phase of the experiment, the conditioned reaction is connected to the 
response-produced cues mentioned above. Then in the test, the first stim- 
uius evokes the response which produces these cues. They, in turn, call 
out the CR. Although cumbersome and a departure from the traditional 
S-R position this hypothesis has value. It suggests certain experiments, 
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Advances )n psychophannacology and experimental instrumentation now 
make it possible to immobilize an animal s skeletal musculature com 
pletely , without interfering with the sensory mechanisms Under these 
conditions associations between neutral stimuli and shock are readily 
established, and later on m a normal state the animal makes appropriate 
avoidant responses The threat to any theory which demands the occur 
rence of a response in the sense of overt si eletal motion is obvious 
6 Latent Learning The term latent learning refers to the fact that 
under certain circumstances animals apparently can acquire habits 
which are not immediately translated into performance either because of 
the absence of reward or because the animal is not motivated for a re 
ward which is present Investigators have performed experiments of five 
different kinds m the study of latent learning In brief summary, these 
are 

a Trials without reward, followed by trials with reward present 
b Free exploration of a maze without reward followed by tnals with reward 
present 

c Trials with rewards present and the animal satiated, followed by trials 
under the appropriate dme state 

d Trials under one drive (such as thirst) with appropriate reward (water) 
under conditions where an unwanted reward (e g food) is present at one 
place in a maze Tests are performed later when the animal motivated by the 
dnve which was originally satiated (eg hunger) 
e Exploration, followed by reward m a distinctive goal box, and then a senes 
of trials in the maze 

All of these procedures except for method ( d ) have provided substan 
tial evidence for latent learning Attempting to explain this effect S R 

effect theorists again have turned to versions of the r G > So mech 

amsm The explanations thus generated seem successful for certain forms 
of the latent learning experiment 

The effect of the argument and debate which has been carried on m 
connection with these problems seems to have been about as follows In 
its preoccupation with the disproof of the S R theory S S theory has 
actually contributed to the development of the position it set out to de 
stroy In the face of criticism from the cognitiv e theorists the S R the 
onsts were forced to develop new concepts new ways of looking at old 
problems and new experimental approaches With such theoretical and 
experimental advances the general appearance of learning theory has 
now changed in two important ways (1) multiprocess views are held 
much more commonly than they formerly were and (2) interest has now 
shifted to a set of problems of much more limited scope The problem of 
the nature of reinforcement is still central to these newer interests, but 
probably no one any longer expects to discover a single simple reinforce- 
ment mechanism Wc turn next to the newer conceptions of reinforcement 
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Mechanisms of Reward 


When it is exaiiixed more closely than we have hitherto paused to do, 
the remioreement occasion seems to consist of three distinguishable as- 
pects involving (1) the presentation of a stimulus to which the organism 
(2) responds in a vray which typically contributes to (3) tension reduc- 
tion of some sort. Such an analysis naturally raises a question: Is one of 
these the critical event in the reinforcement process? This question has 
been asked at least implicitly in the formulation of the never conceptions 
of the nature of reinforcement vrhieh now have replaced the classical 
views discussed in the last chapter. 


CONTEMPORARY VIEWS OF REINFORCEMENT 

Modem conceptions of the nature of reinforcement differ from the 
ear er ones in being less general and in being oriented more concretely 
toward the specific factual questions suggested by the analysis of rein- 
roreement^ outlined above: Is the basic nature of reinforcement to be 
ound m its stimulus aspect, in consummatory behavior, in tension redue- 
on, or m some combination of these components? In keeping with this 

“rs approadi to tbe probiem ’ there 13 ^ tendency than * 

Sh Sot d T enmentS C ° nCeived 111 one tra dition to clash head-on 
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suggested above. ° 1 01 tlle component aspects 


Tension Reduction 

regarded as roughly synonymous. 
23S 
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And, for many psychologists, a “reinforcement theorist” is simply an 
individual who accepts some version of the tension reduction interprets 
tion of reinforcement, a “non reinforcement theorist” is someone who 
does not It may be well to comment on this matter before proceeding 

fU As we have seen, reinforcement is basically an empirical term which 
covers the fact that there are conditions (reinforeers) which, when used 
appropriately promote learning The set of conditions which function 
m this way, however, is a very heterogeneous one ™ 
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an intervening variable whose value depends upon specific drive con- 
ditions, C D , many of which correspond closely to need-establishing opera- 
tions. In Hull’s own writings the distinction between drive and need was 
never developed with complete sharpness ; there was a tendency to use the 
terms drive reduction, drive-stimulus reduction, and need reduction more 
or less interchangeably. 

N. E. Miller and his students, on the other hand, have emphasized the 
distinction between need and drive because both Miller’s theory and cer- 
tain of his experimental interests have brought the importance of the 
distinction into sharp focus. For Miller, needs are conceived, as they were 
by Hull, in terms of environmental lacks or excesses which, if uncor- 
reeted, lead the organism or the species to perish. Thus, an organism can 
be said to need water in the same sense as an automobile can be said to 
need water. If the lack continues, normal functioning collapses. Drives, 
on the other hand, are conceptualized mechanistically, rather than ab- 
stractly as Hull did, in stimulus terms: ‘-‘A drive is a strong stimulus 
which impels action. Any stimulus can become a drive if it is made strong 
enough . . . The faint murmur of distant music has but little primary 
drive function; the infernal blare of the neighbor’s radio has consider- 
ably more. (Miller and Dollard, 1941). Reinforcement in this theory 
is conceived as drive-stimulus reduction. Although this position does not 
deny the existence of need states or their biological importance, it lays 
little stress on needs in the definition of reinforcement. 


Consummatory Behavior 

learrdrio- 33 see bi more detail later, many situations where 

reduction Tims ’a 6 a ^ arep ‘ t absence of anything resembling tension 
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did not occur, these investigators argued that their procedure could have 
led only to an increase in tension and that the running response was fol 
lowed by the opposite of drive reduction The results of the experiment 
appear m Figure 9 1 , data for the two groups of noncopulators are com 
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stimuli, sights, and sounds seem, on common-sense grounds, to be rein- 
forcing. There is also some suggestive experimental evidence (Ribble, 
1944) that human infants respond pleasurably to such stimulation, and 
Harlow (1958) presents data which seem to show that tactual stimulation 
may provide the primary basis upon which the infant monkey develops 
an affection for its mother. 

Experimentally the distinction between the reinforcing-stimulus posi- 
tion and the one stressing eonsummatory responses is not easy to make, 
because most of the stimuli which have been used as reinforcers elicit be- 
havior. Whether it is their function as stimuli or the behaviors they evoke 
• which leads to their value as reinforcers is a question which requires for 
its answer more evidence than is now available. 


NEED REDUCTION AND THE PHYSIOLOGICAL 
LOCUS OF REINFORCEMENT 

Research on mechanisms of reinforcement most often has taken the 
drne- or need-reduction theory as a point of reference and treated the 
outcome of experimental studies as confirmations or disproofs of this posi- 
tion. One particularly important aspect of this line of investigation in- 
volves the search for the physiological bases of reinforcement. In such 
studies experimenters have posed questions of two different general kinds 
and have conducted different sorts of experiments in an effort to obtain an 
answer to each. The questions are these: (1) Are there physiological ef- 
ects without known drive- or need-reducing value which are capable of 

V there are ’ ^ evidence calls drive- and need- 
se^V Tt qUeSti ° n - (2) IS driYe ° r need reduction, without 
leamhLf " COns 7 matoi T aeti ^y, capable of producing 
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Mouth and Stomach Factors in Appetitional Reinforcement 
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FIGURE 9.3. Gastric secretion in the stomach. Panel A depicts the pattern of 
secretion under normal conditions. Panel B is a plot of data obtained after 
introduction of food directly into the stomach. Panel C is for data obtained 
o owing sham feeding. Panel D is the sum of the functions presented in 
ane s B and C. The similarity of the pattern of secretion in Panels A and 
D suggests that the total effect results from the addition of two more elemen- 

SloT U9l7)° 1Vlng eVCntS “ Stomach and in the mouth. Data from 


panel), again in a lessened degree, with some indication that the peak of 
the (motion is reached earlier. The fourth panel in Figure 9.3 shows the 
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that this essentially reproduces the total effect of normal feeding upon 
gastric secretion. 
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of these, Kohn (1951) began by training rats to press a panel an a Skin- 
ner type apparatus for a liquid food on a three minute fixed interval 
schedule Then he operated upon the rats and placed a permanent tube 
mto their stomachs so that it was possible to insert liquids there directly 
Regular oral ingestion was also possible Finally, at different times, he 
tested the rats in the panel pushing situation under three different condi 
tions designed to alter the animals’ drive state in different ways One of 
these tests was performed following the oral ingestion of 14 cc of milk 
(mouth milk) In a second condition, 14 cc of milk was placed directly 
into the stomach via the tube, (stomach milk) This and the mouth milk 
condition should have reduced drive to approximately the same degree, 
although m the stomach milk condition the normal ingestion process was 
bypassed The third condition, instituted as a partial control for the effect 
of stomach distension, involved the placing of 14 cc of saline solution 
directly into the stomach (stomach saline) These treatments occurred 
just prior to a retest to determine their effect upon panel pressing rate In 
responses per minute the performances of the rats following these three 
treatments were after mouth milk, 5 64, after stomach milk, 7 34, after 
stomach saline, 13 03 The only reasonable interpretation of these results 
is that the bar pressing rates reflect the drive states of the animals Spe 
cifically it seems that food introduced directly into the stomach decreased 
drive to some extent, and that drinking the milk decreased it slightly 
more One aspect of these results is of particular interest and importance 
The effects just described were obtained within 20 minutes of the ad 
ministration of food This is too soon for any important nutritive effect 
to have occurred This points to the necessity for a sharp distinction he 
tween drive and need Apparently in this study, and m other similar 
ones (Berkun, 51 L Kessen, and Miller, 1952, Miller, Samphner, and 
Woodrow, 1957), drive reduction occurred in the absence of need 
reduction 

Although results of the sort obtained by Kobn seem to indicate that 
nutrients delivered directly to the stomach can reduce drive, they do not 
show. that such drive reduction can function as a reinforcer for learning 
Evidence favoring such a conclusion is, however, available Miller and 
M L Kessen (1952) trained rats in a T maze with three different kinds 
of reinforcement (1) injection of 14 cc of milk for a correct response 
and 14 cc of saline for an incorrect one, (2) mouth reinforccmeut bj 14 
cc of milk for a correct response and 14 cc of saline for an incorrect 
one, (3) mouth milk and saline again hut after a delay, 7 minutes and 
35 seconds, which was the amount of time required for the injection of 
substances directly into the stomach All of the rats learned The group 
reinforced by mouth immediately was best, but the other two proups 
learned The results of the experiment appear m Figures 0 4 and 0 5 for 
number of correct responses and speed of running, respectively 
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FREE CHOICE TRIALS 

FIGURE 9.4. Percentage of correct choices to the side reinforced with milk 
on free-choice trials (N. E. Miller and M. L. Kessen, 1952). 
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The reason for presenting: both of the measures obtained by Miller and 
lessen is to call attention to an important difference Although the rats 
reinforced by stomach, injection attained 100 per cent correct responses, 
their speed of running hardly improved at all A possible interpretation 
suggested by this discrepant effect upon number of cornet choices and 
running speed is that the influence of the injection may be much more 
upon the habit mechanism than upon incentive This speculation follows 
from the fact that speed measures depend much more upon motivational 
(incentive) factors than percentage of correct responses do It is as if the 
animals “knew” where reinforcement occurred, but were not very 
“eager” for it 


Blood Sugar Level and Reinforcement 
It is well known that starvation leads to a lowering of the level of 
sugar in the blood Although this is not the complete physiological basis 
of hunger, it apparently is an important part of it and raises an interest 
ing question about food reinforcement What would happen if blood 
sugar level were to rise immediately after the performance of some par 
ticular response by a hungry animal f Would such “reinforcement” lead 
to a strengthening of the response in question® If it did, the suggestion 
would be that need reduction is a sufficient, if not necessarily an essential, 
condition of reinforcement Several experiments of this sort have been 
reported H W Coppock and Chambers (1954) showed that injection of 
a 10 per cent glucose solution into the tail vein was capable of reinforcing 
a head turning response in rats Chambers (1956a) attempted a similar 
experiment with dogs, but was unsuccessful, probably because the ap 
paratus he used was poorly suited to this particular species In the same 
study he was successful in reinforcing a response in rabbits with mtra 
venous glucose injections The apparatus was a cylindrical enclosure with 
a number of plates on the floor which, when pressed, closed electrical 
connections which recorded the response and in some cases, administered 
reinforcement After a preliminary determination of preference for the 
different floor plates, the rabbits were divided into two matched groups 
For one group (glucose group) intravenous glucose was injected for a 
response to the least preferred plate, intravenous xjlose (a nonnutntne 
monosaccharide) was injected for a response to the preferred plate, and 
a saline solution was injected for a response to anj of the other 11% e plates 
For the other group (xylose group) xjloso was paired with the nonpre 
ferred plate and saliue was paired with the other six Different groups 
were tested under 0, 3, 4, and 6 days of starvation 
The results of the experiment appear in one form in Table 17, which 
shows the proportion of time spent on the rewarded (preuouslj non 
preferred) plate bj each group, together %%ith the difference between the 
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Proportion of Time Spent by Glucose-Reinforced and 


Xylose-Reinforced Rats on the Reinforced Plate 

(Prom Chambers, 1956a) 


Days of 
Starvation 

Mean for Glucose 
Group 

Mean for Xylose 
Group 

Difference 


0 

32.00 

22.23 

9.77 


3 

52.75 

26.55 

26.20 


4 

64.27 

31.95 

32.32 


6 

56.08 

35.78 

20.30 



two groups on this measure at each hunger level. Statistically speaking, 
this difference was significant for the deprived groups, but not for the 
satiated group. This shows that intravenous glucose is capable of reinforc- 
ing a response under conditions of hunger, where the reinforcement is 
appropriate to the drive state, but not otherwise. 

At face value, the results of this experiment seem to provide strong 
support for the need- or drive-reduction view of reinforcement in that 


learning has been produced without consummatory behavior and without 
direct taste stimulation. As Chambers points out, however, the taste 
mechanism cannot be ruled out completely, because the injection of cer- 
tain substances, including some which are chemically related to those 
used in this experiment, produces a sensation of taste within a few sec- 
onds. Conceivably this could have happened, and the animal could have 
experienced a sweet taste which might be either primarily or secondarily 
reinforcing. Moreover, even if this possibility were ruled out, there re- 
mains a question as to the exact physiological mechanism brought into 
operation by the injection of glucose into the blood stream. One possi- 

this k at tb i, a . em ;.^ rature ^ ncrease follows glucose injection and that 
Chamht, ™ a 7 7 t ’ le - remf ° r ' illg eltei!t - In su PP° rt of tMs hypothesis, 
iniectimi rfinT a 7* " “!. a f °U<™-up study (Chambers, 1956b), glucose . 

parts “the but reliaWe , temperature rise iu certain 

P ody. (See also Chambers and Puller, 1958 ) 


STIMULUS INTERPRETATIONS OF REINFORCEMENT 

make the conception V of r^infir ClUdm + g ° f ^ materials l ust covered, 
tion hard to resist? ^ &t leaSt in P art > a stimulus £unc ' 

resist. Among the considerations leading to this conclusion 

Young (1952)/Y^ngVtheory r hot“ver behaVi0r t0 stimu!ational factors is that of 
to present it here as a theory merelv nf’ C ° V ® TS so mucl1 it would be misleading 
consult Young (1959) which presents n einfo ^ cement - The interested reader should 
several relevant references. ? recent statement of his position and lists 
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are the following (1) As has been shown, taste simulation, and perhaps 
proprioceptive feedback from chewing and swallowing, seem to contribute 
to the reinforcing value of food (2) Kemforcers seem to display some 
typically stimulus attributes including absolute and difference tbres 
holds (B A Campbell, 1955, 1956, 1957, 1958, B A Campbell and 
Kraelmg, 1953, Quitman, 1951) (3) Secondary reinforcement is always 
a matter of stimulus presentation (4) Studies presently to be described 
have provided evidence for a reinforcing effect of stimuli when other m 
fluences (such as drive reduction) seem to have been ruled out The most 
important lines of experimentation in this last category are those con 
cerned with the reinforcing effects of (a) taste stimulation, (t) explore 
tion, and (c) electrical stimulation of the brain 


Taste Stimuli as Reinforcers 

Up to a point, rats prefer a mild salt solution to plain tap water and 
show a peak preference in the neighborhood of the concentration of 


PRE-TEST 



PERIOD LOS % CONCENTRATION 

NoGI SOLUTION 

FIGURE 9 6 Graph showing the preference of the white rat for salt solutions 
ol various strengths The pre test period provided a measure of the amount 
ol plain water consumed from each of two Lotties (Bare, 1949) 

physiological saline, 9 per cent, as is illustrated m the intake functions 
m Figure 9 6 These facts hate been demonstrated nnny times {Hire 
1949, Kichter, 1939) From the point of wen of reinforcement theory, 
the two most important considerations brought to light bj research in 
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this general area are the following: (1) Preference for mild salt solutions 
appears in animals whose diets contain enough sodium chloride for nu- 
tritional purposes and, thus, are in no state of need. (2) Salt deficiency, 
when it occurs, as with adrenalectomy, increases salt intake and may 
lower the preference threshold for salt solutions over plain water. This 
latter fact points to the importance of nutritional variables in the control 
of the rat s salt intake and is in line with a drive-reduction view. 2 For 
our present purposes, the former result is of greater interest, for it seems 
to mean that rats like the taste of weak salt solutions, and raises the 
question whether taste preferences alone might not provide an adequate 
basis for reinforcement. The most extensive line of evidence available for 
consideration in attempting to answer this question involves the results 
of experiments in which saccharine, a nonnutritive sweet substance, has 
been employed as a reinforeer. Sheffield and his associates have per- 
formed a number of such studies. In one (Sheffield and Roby, 1950), it 
was shown that rats preferred a .13 per cent saccharine solution to tap 
water and that such a solution, used as a reward in a T-maze, led to a 
performance which was nearly as efficient as that obtained with food re- 
ward, whether measured in terms of number of correct responses or in 
terms of error-reduction with practice. A later study (Sheffield, Roby, 
an Campbell, 19o4) showed that the same saccharine solution was a more 

effective reinforeer m a runway situation than a nutritive, but less sweet, 
dextrose solution. 

italf t" faC V f U ’ SUCh reSUltS Seem t0 prove that a sweet b y 
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survive insulin shock if thev f a ^ . Were more llkel y to 

insulin injection than if they were not mpT® .^ e f* tely after the 
p. 556n.). This fact ™intc + ® mitb an( I Capretta, 1956, 

and sugar metabolism. If as this scanty 6 between saccharine 

have some indirect ““ 3065 

a j sm or on hunger, there remains 
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cient rats, which had been deprived of tho ’ ^ ese lnv estigators tested sodium- 
o learn the location of a salt solution*! t taSte ^^ 7 , for the ability 

earning, indicated that reduction of the sodium' Z £ound some evidence for such 
remforcer - e SOdlum need ma y function as an adequate 
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at least tlie possibility that the reinforcing effect of saccharine is trace 
able to a dm e reduction process 

Experiments investigating the effect of hunger upon responses rein 
forced with saccharine and other sweet tasting substances are relevant at 
this point, because, if saccharine does have some indirect metabolic effect 
it is at least possible that the reinforcing power of the substance would 
change with the level of this drive Sheffield and Roby (1950) and 31 
Smith and 31 Duffy (1957a) both have found that hungry rats m a free 
drinking situation consume more saccharine water than satiated rats 
There is, however, no good evidence that this effect carries over into more 
typical learning situations Using sucrose, which also apparently secures 
its reinforcing value mainly because of its sweet taste (Young and Shu 
ford, 1954, 1955), 3f Smith and Kinney (1956) obtained no difference in 
bar pressing rates for hungry and satiated rats reinforced with sugar 
water This is to be contrasted with the very great effect that the sweet 
ness of the solution has on bar pressing rate (Guttznan, 1953) Thus, there 
13 reason to believe that it actually is the taste of sugar which provides 
it with its reinforcing power 

Even accepting this conclusion, however, a different interpretation of 
the results of experiments in this area is possible, in terms of secondary 
reinforcement It could be argued that certain tastes, such as sweet tastes, 
are reinforcing because, m the past, they have been associated with eating 
and have acquired reinforcing power this way (Bare 1949) Against such 
an interpretation there stands first of all, the fact that Sheffield and 
Roby observ ed no diminution of the reinforcing power of saccharine, al 
though they used the same rats in an extensive series of experiments and 
for a long period of time in their study If the reinforcing power of 
saccharine were entirely a matter of secondary reinforcement, there 
should have been some evidence of extinction On the other hand 31 P 
Smith and Capretta (1956) ba\e reasoned that the Sheffield and Roby ex 
penment did not meet the conditions which would be necessary to extm 
guisk the secondary reinforcing power of saccharine because of the 
possibility that the sweet taste of saccharine and digestive processes oe 
curred together during some of the experimental tests More specifically 
if the animals were allowed to drink saccharine while the previous^ in 
gested food remained in the stomach, this would provide the opportunity 
for the taste of saccharine to be associated with drive reduction Smith and 
Capretta proposed that saccharine would lose its secondary reinforcement 
value only when such an association with drive reduction or need redue 
t;on had been completely eliminated Pursuing tins hypothesis, they per- 
formed an expenmeut with six groups of rats which received 4, S, or 15 
one hour exposures to saccharine after 2 or 21 hours of food deprivation 
Following such training there was a test using a T maze to determine the 
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reinforcing power of saccharine in a learning situation. Two major re- 
sults came out of this experiment. The first was that the animals deprived 
of food for two hours seemed to develop a stronger preference for sac- 
charine than the hungrier animals which had been starved for 21 hours. 
Figure 9.7 shows this result in the form of curves depicting the decreas- 
ing amount of time required to consume 8 ml. of the solution. The second 
result was that, in the T-maze tests, in which all subjects had been with- 



the 2-hour and 21-ho^™roups^ 1 revious drink ^ CC ' ° f Saccharine solution. For 
under 2 or 21 hours of ^d been 

lively, this experience extended over 4 8 n i c a and C grou P s ’ respec- 
Capretta (1956). ’ a or 15 da y s - Data from Smith and 


charine had been Sr^ho^of fTodd eSperienee ^h sac- 

the other group. This ohenomorm * i epriva t 10ri learned better than 

Both of these results indicate that saceha 6 gr ° Ups m the T-maze test. 

that saccharine was a more powerful rein- 
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forcer for the animals whose original experience with saccharine had ht 
under low drive Moreover, Table 18 clearly indicates that mcreasi 
amounts of experience with saccharine produced opposite trends in 1 

TABLE 28 

Mean Errors in T Maze tor Each Saccharine Group 


(From M P Smith and Copretta, 1956) 


Days of Experience with Saccharine 

Sours of Depnxation 

2 21 

4 

13 75 

13 88 

8 

1213 

14 00 

15 

10 25 

16 12 


two sets of deprivation groups an increase in the reinforcing power o 
saccharine (fewer errors) for the 2 hour group, a decreasing- etfectiienes 
for the 21 hour group This last fact suggests that the taste of saccharin 
loses some of its power (extinguishes) when it is presented when the am 
mal is hungry and not rewarded If so, this means that at least a part oi 
the reinforcing power of saccharine is learned On the other hand, If 
hours of extinction was incapable of completely destroying the reward 
value of saccharine Perhaps more experience would have had such an 
effect, as the trend displayed by the errors for the three 21 hour groups 
suggests It 25 equal Jy conceivable, how ever, that the taste of saccharine is 
primarily reinforcing and that the secondary reinforcement component is 
very small 

Reinforcement by Exploration 

The thesis that reinforcement occurs only as a result of drive reduc 
tion has also been called into question by tile outcomes of experiments 
Winch show that animals will learn for a reward which is apparent^ 
nothing more than the privilege of exploring a complex environment Tor 
example Montgomery (1954) ran rats in a Y maze m which one arm of 
the Y terminated m a typical goal box and the other led to a Daslnell 
type maze which the rat could explore After 24 trials with the Daslnell 
maze on one side, the conditions were reversed and the Daslnell maze was 
placed on the other side The animals learned to go to the side containing 
the Dashicll maze and reversed their preference with a change m coiidi 
tions In a later study Montgomery and Scgill (1955) showed that rats 
could learn a black nlnte discrimination for a similar reinforcement, mid 
A K Meyers aud N V Miller ha\c shown that rats mil learn to press a 
bar m order to explore a compartment to which the bar pressing response 
gams au entry The threat to dm c reduction theory implicit m such 
findings is obvious It looks very much as if a response « acquired with 
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the arousal of an exploratory motive rather than with its reduction 
(Montgomery, 1954). 3 

Even more impressive results have been obtained in studies with mon- 
keys. In one experiment, Butler (1953) tested monkeys in an enclosed 
box located in a large room in which there usually were people engaged 
in one sort of activity or another. The box contained two windows with a 
provision for putting a blue card in one window and a yellow card in the 
other and for changing the positions of the two cards from side to side. 
Pushing on one of the windows opened it, allowing the subjects to look 
out and observe the happenings in the room. All of the animals learned 


to open the door with the card which signified that it could be opened and 
kept at the task with great persistence. An average of 40 per cent of an 
animal s time may be spent in visual exploration of a situation if the 
choice is left up to the subject (Butler and Alexander, 1955). In later 
studies Butler showed that different incentives have different values. The 
sight of another monkey is possibly the most effective visual incentive; 
but the opportunity to watch an electric train, to observe a tray of food, 
or just to observe an empty box leads to learning (Butler, 1954). More- 
over, the mere sound of a train running and the noises made by the other 
monkeys in the colony are also adequate reinforeers (Butler, 1957a). 
About the only kind of stimulation which a monkey will not learn to pro- 
. 1S a fear - ellc dmg one such as a large dog (Butler, 1958a, 1958b). It 
is of some interest that the effectiveness of visual reinforcers increases 
with up to 6 or 8 hours deprivation from the opportunity to engage in 

u° n (B + Utler ’ 1957b )' In this regard, the exploratory motive 
resembles the homeostatic drives. 
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1955; Hurwitz, 1956; Kling, Horowitz, and Delhagen, 1956; C. h. 
Roberts, Marx, and Collier, 1958). 
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rules out the most obvious interpretation in terms of secondary reinforce- 
ment by showing that the reinforcing value of light exists over and above 
any i nfl uence producible by associating light with food and eating. (2) 
At tbe same time, it indicates that secondary reinforcement may play a 
subordinate role in that there is an interaction between maintenance illu- 
mination and the effect of the specific reinforcer. 


Reinforcement by Brain Stimulation 

By far the most spectacular evidence in favor of a stimulational theory 
of reinforcement is to be found in the discovery of numerous areas in the 
lower brain where weak electrical stimulation is capable of reinforcing 
behavior. Since the work of Olds and Milner (1954) 4 the phenomenon 
has been a matter of very great interest to psychologists. In their demon- 
stration of this effect, Olds and Milner used as subjects rats that had 



FIGURE 9.9. Schematic 


diagram to represent brain-stimulation 
(Olds, 1956). 


apparatus 


(1956), Oias 6 (1958) b ’ 19 ° 6C ’ 1956d ’ 1956e )> 01ds , Killam, and Bach-y-Rifc 
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electrodes chrontcally implanted m the limbic ^ % 

startling fact was that ma Y ra * pr , |e o£ sat , a tion, ceasing to 

obtain this stimulation and showed = exhausted (Olds, 1958b) 

press the bar only after becoming phys C f"““ ve ral dlys It is 
Figuie 910 shows a record w “'“ enness wlth w hich bar pressing 
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Mppoc^mp^ nu f clear mass of the thalamus, or 

stimulation. Delgado, Rosvold, and Looney TigseT ^ t0 aV ° id T* 
conditioned a fear response to a v n A 7 ( 19 ° 6 ), using monkeys, first 

shock to the feet. In ZS of t» t by pa ™ g Painful 

senes of testing trials brain stimulation teas some- 
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times substituted for the tone, -with the result that the stimulation of cer- 
tain areas led to an avoidance response indistinguishable from that 
elicited by the original CS The areas reported as eliciting fear when 
stimulated were “medial nucleus of the amygdala and adjacent tissue m 
the rhinal fissure, trigeminal nerve at the Gasserian ganglion, rostral 
part of the pons, medial part of the mesencephalon in the vicinity of the 
central gray, nucleus ventralis posteromedialis, external part of the nu- 
cleus ventralis posterolateralis of the thalamus, and external medullary 
lamina of the pallidum ” (Delgado, Eosvold, and Looney, 1956, p 379) 
In another demonstration that brain stimulation may serve as a basis for 
avoidance learning (Cohen, G W Brown, and M L Brown, 1957), cats 
learned to run to avoid brain stimulation delivered to the anterior hypo 
thalamus or to the median forebrain bundle when the onset of such stimu- 
lation was signalled by a tone Thus we see that brain stimulation may be 
used either as a positive reinforcer or as a negative one which apparently 
elicits pain or fear The question which naturally arises concerns the con- 
ditions under which brain stimulation is a fear elicitor and under which 


it is a positive reinforeer It seems likely that one important factor would 
be the exact placement of the electrode There is as yet, however, no defi- 
nite mapping of the areas which are positively and negatively reinforcing 
Two characteristics of the shock, itself, are known to be important 
parameters in determining the effect of brain stimulation They are shock 
intensity and duration 5 In his early work. Olds (1955) reported that rats 
seemed to be positiv ely reinforced by all intensities of brain stimulation, 
up to and including intensities which produced convulsions More recent 
evidence suggests that high intensities of stimulation are, at least, less 
effective than lower intensities, and there is some evidence that still 
higher levels may become aversive Reynolds (3958) studied the rate of 
bar pressing for brain shocks of 5 second duration and varying intens- 
ities delivered to the hypothalamus A sample set of results, shown in 
Figure 9 11, indicate that increasing voltage is increasingly reinforcing 
up to a point and that, beyond this point, the stimulus is less so For dif- 
ferent rats the most effective voltage was from 1 9 to 3 8 volts At higher 
shock intensities behavior appeared which might be interpreted as a re- 
sponse to pain, it included withdrawal from the bar and (sometimes 
successful) attempts to escape from the apparatus The bearing of this 
upon the function of brain stimulation is that the experiments obtaining 
avoidance learning With brain stimulation have usually employed fairly 
high levels of shock Cohen, Broun, and Broun, for example, set tlwir 
shock at a level capable of eliciting certain emotion like reactions from 
the cat including hissing, spitting, pilo erection (hair standing on end), 
pupillary dilation, and aimless jumping The intensities wtre 1 2 ma ami 


• Reinforcement schedules also influence lbc rate of responding for Inin 
tion (Sidman, Brady, Boren, Conrad, and bchulmaa 1955) 
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STIMULATION VOLTAGE 
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bos to turn off (escape) the team shock , hut only the group which showed 
the alarm reaction learned to avoid the team shock by responding antic, 
patently To Roberts this suggested that the brain stimulus may have had 
a dual effect He reasoned that, for the annuals showing the fl.gh 
reaction, the shock was probably pos.t.vely reinforcing at its onset but 
when continued, it gradually became painful If this analysis is eorrect r 
means that the animal was first reinforced by shock onset for doing noth 
"en atr the shock had been on for long 
the animal was reinforced for an escape response To 
Roberts (1958b, 

ttehalZs 
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themselves are that brain stimulation might produce pleasant olfactory 
sensations, feelings of satisfaction like those which follow eating or drink- 
ing, or perhaps, sexy feelings either of arousal or satisfaction. But, 
phrased in this way, such speculations obviously violate the most basic 
rules of scientific methodology. They represent an operationally impos- 
sible invasion of the white rats “private experience” and are inaccessible 
to experimental test. 

These methodologically unsatisfactory questions about the psycho- 
logical processes related to brain stimulation do, however, lead to testable 
hypotheses if they are translated into more nearly behavioral terms. One 
might ask, for instance, whether the effectiveness of brain stimulation 
varies with the level of hunger, thirst, or sexual motivation of the animal. 
If it does, this fact would lend credence to the idea that brain stimulation 
is somehow related to these motives. Brady, Boren, Conrad, and Sidman 
(1957) have studied the effect of food and water deprivation upon rate of 
responding for brain stimulation, and Olds (1958a) has obtained evi- 
dence on the effect of hunger and sexual motivation. In this latter study 
sixteen male rats were first trained to press the bar in a Skinner ap- 
paratus for brain shock, hungry on some days and satiated for food on 
others. Then they were castrated to produce a low androgen (sex hor- 
mone) level and tested for rate of bar pressing under this condition which 


was assumed to produce a low level of sex drive. Finally, sexual motiva- 
tion was restored by injections of testosterone propionate, and bar press- 
ing for brain stimulation tested again. Depending upon the location of 
the electrode, increasing intensity of hunger or of the sex drive either 
e eva e t e rate of bar pressing, or depressed it. Moreover, there was a 
negative relationship between the androgen and hunger effects. The brain 
locations which, when stimulated, led to the greatest increases in response 

eve under hunger suffered a relative decrease with a high androgen 
level, and vice versa. 
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IV -T« 1Q531 (2) The fact that electrical stimulation 
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these investigators to adopt consummatory response theory of reinforce- 
ment and to attempt to demonstrate the reinforcing power of other 
activities, such as copulation without ejaculation (Sheffield, Wulff, and 
Backer, 1951). Kagan and Berkun (1954) found that rats learned to 
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tion drive, Harlow and McCleam (1954) showed ‘hat 
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TWO-PROCESS THEORIES OF REINFORCEMENT 

If there is one most important conclusion to be drawn from the mass of 
evidence on the mechanisms of reward, it would seem to be that no single 
theory, as currently formulated, can cover all of the data. The experi- 
ments described in the previous sections succeeded in demonstrating the 
partial validity of several hypotheses, without seriously invalidating 
others. The inference to which this situation almost inevitably leads is 
that reinforcement must be much more complicated than the simple for- 
mulations of the single-process theories imply. Such a conclusion natu- 
rally leads to the idea that the nature of reinforcement may d iff er from 
one kind of learning situation to another. Theories which take this general 
point of view are called multiprocess theories. 

Of the multiproeess theories, the most popular is a two-process theory 
v.hieh maintains that a tension-reduction explanation applies to instru- 
mental learning and that a contiguity principle, like that described in the 
last chapter, applies to classical conditioning. The origin of this two- 
process position is in certain difficulties which arise when the learning 
theorist, especially the drive-reduction theorist, attempts to explain learn- 
ing based on noxious stimulation. 8 The first discussion of the problems of 
negative reinforeers in a line which links directly to present-day accounts 
of escape and avoidance learning was by Hull (1929). 7 In this early 
paper, Hull outlined a very simple version of the theory which has since 
dominated learning-theoretical discussions of avoidance behavior. He 
also called attention to some of the problems which such behavior poses 
for the learning theorist. Arguing from a generally functionalist position, 
Hffil emphasized the survival value of defense reactions and called atten- 

S th ° ? o r eC T ty i 0CCUrrence of ^ese responses in anticipation 

illTir ° e aVOided - Then he P r0Ceeded to point out the 
reaoXn tn h ^ „ gr ° W ° Ut of this fact. Specifically, a defense 

inextinguishable; but if it is, 
tinction and t,T ! generality of the principle of experimental ex- 

STr^i ns Vnlnen * te to the acquisition of de- 

reactions to all sorts of stimuli which accidentally happen to have 

discussed .ere developed in connee- 
tlus contest, evidence is beginning to accumulating almost exclusively within 

situations involving positive reward. The theorv wh h fruitfuU y extended to 

be converging upon parallels that for avnirl , ch . soveral investigators seem to 

component stemming from an association ■hot-n- 6 earnin " : _ classically conditioned 

a motivational element (incentive') e , e . n neu ^ ra ^ stimuli and reward provides 

reward (Powell and Perkins 1957- w, mot ^ ates performance and also provides 
Siskel, 1959). Mowrer (S) WntlvTn ’ 19a1 ’ 1952 5 S P enee > 1956 5 Collier and 
some detail. V ; eUtl> has Presented such a theory, worked out in 

Dinsmooi -0 the role of negative reinforcement see 

U9o3a ’ 19o3h > 19 ^a, 1951b, 1955a, 1955b). 
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subprocesses assumed to take place in avoidance learning. In this experi- 
ment, rats were first trained to escape from shock by crossing a barrier 
in a shuttle box. After they had learned this, the rats were confined in a 
limited area of the shuttle box and given a series of unavoidable shocks 
which were paired with a buzzer, after the manner of classical condition- 
ing. Finally, with the escape response again possible, the buzzer was 
sounded to determine whether, in the absence of shock, the rats would 
cross the hurdle in response to it. In two slightly different replications of 
this experiment, approximately 80 per cent of the animals responded to 
the buzzer by crossing the barrier. Fewer than 10 per cent of control 
animals which had received buzzer only, shock only, or unpaired buzzer 
and shock in the second phase of the experiment responded in this way. 
Obviously the pairing of buzzer and shock w T as crucial in producing es- 
cape behavior on the test trials. 

The two-process interpretation of the learning which occurred in this 
experiment may now r be presented with the aid of a series of diagrams: 

1. Training to Escape Shock 


S (shock) » r x » s x (fear) > R (escape) 

WtL lntenti0n no tlliS 1 diaSram ^ t0 Shmv two thin gs: (a) that shock, 
canlbl 7 r CS 7 aS an eUcit0r ° f eSCape bebay ior, soon becomes 
through fr,T ng 7 v CaPe reSp ° nSe ’ indica ted by the upper arrow, 
tonal IT 777 conditioning, and (6) that shock also produces in- 

uH at 7 77 7 Wlth COnse ^ ent stimuli, s 5 . These internal stim- 

tan escTneTrl7 t ^ ^ " escap * f -m them, rather 

learning In 77-t 7 T ^ Ch COnstitutes the reinforcement for this 

a cTfoth/7 functioning as a drive stimulus, s x also becomes 

a tor the escape response. 

2. Fear Conditioning-. Buzzer Paired with Shock 

Shock > r „ r 

1 * s * l 110 escape response possible) 

Buzzer'' 

theory assumes thatoon'touityt the C ° ndi . ti0ning - Tw °-P roeess 

for such conditioning to occur. and * x 1S a11 that 18 re( l uirecl 

3. Test Trials : Buzzer hut no Shock 

Buzzer > r 

metanC'l the internal 

(&) the buzzer is capable of eliciting eUcitlng the esca P e reaction, and 
forms the escape reaction in roc ° 1 * Sl ' ^ bus the animal per- 

of importance to be made in tM^^^' s T here are two point8 

section, (a) Since s x is capable of 
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evoking the escape reaction, it piovides a potential explanation for the 
occurrence of avoidant behavior in anticipation of a noxious stimulus 
(6) Since s x is assumed to have drive properties, there is also a device for 
the reinforcement of avoidant behavior through the elimination of s x 
In ordinary avoidance conditioning studies it is assumed that the three 
phases outlined above all occur together On early trials, the warning 
signal and the stimulus to be avoided occur one after the other and cou 
tmue to do so until the avoidance response appears During this stage the 
animal is learning to escape from the noxious stimulus (stage I above) 
and also acquiring a fear of the signal (stage 2 above) When the avoid- 
ance response finally appears, it is elicited by response produced stimuli 
and reinforced by escape from the CS (Miller, 1948) This corresponds 
to stage 3 in May’s experiment described above 


The Nature of the Fear Response 
The most important questions raised by the two process theory of 
avoidance conditioning concern the details of the fear conditioning proc 
ess One of the specific problems involves the nature of the classically con 
ditioned fear reaction Mowrer (1947) inclines toward the belief that the 
fear responses are exclusively autonomic I? L Solomon and Wynne 
(1954) believe that the response is more complicated, consisting of rcac 
tions of the autonomic nervous system winch result in visceral responses 
of high magnitude, skeletal motor discharge, neuroendocrine responses, 
and activity of the higher central nervous sjstem This view is based m 
large part upon the results of an experiment (Wynne and It L Solomon, 
1955) on avoidance learning in sympatbeclomized dogs These animals, 
whose autonomic processes were effectively blocked learned to avoid 
shock, although with greater difficulty than normal dogs Moreover, dogs 
deprived of autonomic activity after avoidance conditioning were, if anj 
thing, more resistant to extinction than normal dogs These facts appear 
to require an extension of Mowrer ’s interpretation to include these other 
processes, perhaps in the direction of nonmuscuhr neural responses The 
most recent statements of the major learning theorists (Ivoeh 1959) 
seems to indicate that many of them current!} favor this conception 


Reinforcement for the Acquisition of Fear 
As regards the acquisition of fear itself, two-process tlicorj makes two 
assumptions (2) that fear is established hj a process of conditioning and 
(2) that tins conditioning is specifically a mutter of cotitiguifj learning 
Of these the first is less a matter of dispute than the second Two process 
learning theory could, as in Hull’s 1929 treatment, proceed without ac- 
cepting cither tko effect or the contiguitj Iheorj for tlic development of 
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crucial test of tlie two explanations, because positive results can be 
handled quite easily from either point of view, and because negative re- 
sults would create problems for either of them. This point 'will be easy to 
make with the aid of an experimental example. Spence (1953), in a 
faetorially designed experiment, first conditioned the eyelid reflexes of 
human subjects, using strong and weak airpuffs as the unconditioned 
stimuli for two different groups. On the second day of the experiment the 
TJCS intensity remained the same for half of the subjects in each group, 
but for the other half, it was switched to the other intensity. The four 
groups thus created produced the numbers of CR’s shown in Table 19 


TABLE 19 

Mean Number of CR’s in First 20 Trials on Day 2 

(From Spence, 1953) 


Bay 1 UCS 

Bay 2 UCS 

weak strong 

Means reflecting 
learning differences 

weak 

5.65 

8.80 

7.23 

strong 

7.45 

13.00 

in 9Q 

Means reflecting perform- 




ance differences (drive) 

6.55 

10.90 



on the first 20 trials of the second day. Our particular interest is in the 
row means. As the results of faetorially designed experiments are usually 
interpreted (p. 118), the difference between these values shows that the 
subjects in the strong UCS group had learned more on the first day of 
framing than the subjects in the weak UCS group and that the effect was 

nofreZTtV^ T i ^ ° f the col ™> on the other hand, can- 

the hritori tit ^ b6CaUSe ’ avera S ed ^oss day-1 conditions, 

means of tbe°e f 818 identicaL Diff *rences between the 
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In sreneral - t • -ui y ’ Tv ^ Len tbese measures were obtained. 

eff eet of ' UCS T * T ******* theSe results as evidence for an 
oftt III perf ™nce. The contention 

learning because of "the greater VeM ^ Str ° nger UCS leads to better 
cessation of this stimulus. reinf orcing value associated with the 

cifically, it mfgbrbVrrgueTth^tbe 11 ^ mterpretations are possible. Spe- 

reflex of greater vigor Sd that b erie ^ day 1 elicited * 

Or, it might be proposed that the Lon^UC ^ C | Urred for this reason ‘ 
stronger fear drive to be ■> x ? UCS on da F 1 created a 

fear was responsible for the superior nVf ' ' ^ and ^ ^ acquired 

performance of these subjects on 
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day 2 Neither of these accounts it will be noted, says anything about 
drive reduction or reward 

In an effort to develop a purer test of the effect of UCS intensity upon 
learning Spence and his students subsequently turned to a different ex 
penmental design In these studies (Ross and Hunter, J9a9, Spence, 
Haggard, and Ross 1958a 1958b) groups were conditioned with uncon 
ditioned stimuli of different intensities In order to equate drive strength, 
the subjects conditioned with the weaker UCS received interspersed trials 
with the strong UCS and those conditioned with the stronger UCS re 
ceived interspersed trials with the weak UCS On these trials there was 
no CS The results of one of these studies appear in Figure 9 14 The 
superiority of the group conditioned with the strong UCS is taken as 
evidence that the intensity of the unconditioned stimulus has an effect 
upon learning and not just performance It should be noted however, 
that even if these experiments demonstrate an effect of UCS intensity 
upon learning they do not prove that duve reduction produced by the 



FIGURE 914 Percentage of conditioned responses obtained with strong (S) 
and weak (W) unconditioned stimuli when interpolated trials with the 
alternative UCS are used to equate experience with the different intensities 
(Rosa and Hunter, 1959) 

cessation of the UCS is responsible for this effect In fact other tests of 
the drive reduction hypothesis earned out in the content of classical de 
fense conditioning cast serious doubt upon such an interpretation 
Sm ce drive reduction theorj equates the cessation of a noxious stun 
ulus with reinforcement this theory predicts poor conditioning with a 
UCS of long duration because in this ease reinforccmuit is deli} ed I x 
penments manipulating duration of the UCS however, have shown tint 
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the magnitude of that variable is of little or no importance in the case of 
the conditioned GSR (Bitterman, Reed, and Krauskopf, 1952), an ac- 
quired fear response in rats (Mowrer and L. N. Solomon, 1954), cardiac 
conditioning (Wegner and Zeaman, 1958), and the conditioned eyelid 
reflex (Runquist and Spence, 1959). Beyond this, drive-reduction theory 
maintains that the noxious stimulus must undergo an abrupt diminution 
for an acquired drive to be learned. More specifically, the rate at which 
a noxious stimulus goes off should affect the strength of fear. To test this 
proposition, Mowrer and L. N. Solomon (1954) paired the same CS, a 
3-second blinking light, with shock under the four arrangements shown 
in Figure 9.15 for different groups of rats. After this training the effect 

CS j 1 

UnCS 1 I 1 


CS. 

UnCS. 


CS. 

UnCS. 
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The Reinforcement of Avoidance Responses 

In the most general sense, it seems that avoidant reactions are rein- 
forced by the cessation of, or escape from, the once neutral stimulus now 
associated with pain One of the most impressive lines of evidence for this 
conclusion is that postponing the cessation of the feared CS beyond the 
moment of occurrence of the avoidant response (delay of reinforcement) 
retards the acquisition of conditioned avoidance reactions (Mowrer and 
Lamoreaux, 1942, 1946) It is also known that there is a delay of rein- 
forcement gradient which resembles that obtained for positive reinforcers 
(Church and R L Solomon, 1956, Kamin, 1957a, 1957b, Wickens and 
Platte, 1954) Looking beyond this most general interpretation, however, 
we are immediately confronted with problems, all of which, in one way 
or another, involve the difficulty of extinguishing avoidant behavior (R 
L Solomon, Kamin, and Wynne, 1953) In the experiments performed by 
Solomon and his associates, the subjects have usually been dogs trained 
to avoid shock in a two compartment apparatus The conditioned stimulus, 
signalling the onset of a very strong (traumatic) shock, was a change in 
illumination and the lowering of a door, which made it possible for the 
dog to jump a shoulder high hurdle and get into the other compartment to 
avoid the shock or escape from it m the early stages of learning The m 
terval between the signal and shock onset was 10 seconds 
Figure 916 shows the results obtained from what Solomon and Wynne 
(1953) describe as a fairly typical subject Note the following facts about 
this dog’s performance (1) After seven trials with latencies longer than 
10 seconds the dog avoids the shock on trial 8 and never experiences it 
again (2) In spite of the fact that there is no shock, the dog improves its 
performance throughout the fifty two remaining trials, rapidly at first 
and then more slowly There is no evidence of extinction here, or of back 
sliding to receive the occasional reminder that the situation is dangerous 
In fact, after 10 successive avoidances m this experiment, Solomon and 
Wynne discontinued the shock altogether This procedure brought certain 
other uapertsat facts to light (3} The oecosieos) king hteney responses 
winch occurred without shock never resulted m extinction, but if any- 
thing, strengthened the animal's behavior on the immediately following 
trials as if such procrastination enhanced the subject’s fear of the situa- 
tion This observation should be coupled with another common one in 
avoidance training, (4) that indices of fear other than the avoidance re- 
sponse itself show a brief period of increase during the first three or four 
trials which is followed by marked decrease later in learning Solomon 
and Wynne describe this fact for several fear indicators m their experi- 
ment It has also been noted in the case of other responses in other species 
(Kessen, 1 953, Slowrer and Lamoreaut, 1951) Adding a fifth fact (5) 
that the hurdle jumping responses m Solomon’s and Wjmie’s studj per- 
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nCURE 9 . 16 . Performance of a single, typical dog in avoidance conditioning 
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other studies, it is not applicable here Such facts as this led Solomon and 
Wynne to propose an elaborated version of the two factor theory to ex 
plain traumatic avoidance conditioning The mam elements in their posi- 
tion are those developed by Mowrer the classical conditioning of fear by 
stimulus response contiguity , and the subsequent establishment of in 
strumeutal avoidant behavior on the basis of dme (fear) reduction 
Beyond this, however, they suggest two additional principles, the prm 
ciple of anxiety conservation and the principle of partial irreversibility, 
which are descriptive hypotheses drawn from their conditioning data 
The typical asymptotic latency of the hurdle jumping response, and 
also of the wheel turning response in rats, is about I 5 seconds This 
latency is less than that of almost all visceral responses and certainly less 
than the tune between the presentation of a feared stimulus and the re 
ception of a feedback from such responses This means that the avoid 
ance reaction takes place too soon for some of the most important elements 
of the fear response to occur And, if the response terminates the fear 
stimulus, these components of the response will never occur m the pres 
ence of the stimulus It will, therefore, be difficult to extinguish the fear 
response Because of its protracted latency, anxiety is in this way saved 
from extinction or, as Solomon and Wynne say, conserved 

On the other hand, occasional long latency avoidance responses do 
occur with two effects The first is that such a response allows the fear 
reaction to occur in greater intensity than on short latency trials This 
increase in motivation accounts for the fact that such responses, even if 
not punished, often lead to a faster reaction on the next trial On the 
other hand, with such responses, there is also an opportunity for the ex 
tinction of the fear response to occur, and it is impossible to a\oid the 
prediction that this will indirectly weaken avoidance behavior and ulti 
mately result in its disappearance To handle this problem, Solomon and 
Wynne offer their second principle, that of partial irreversibility This 
principle, patterned after G W Allport's (1037) concept of functional 
autonomy, maintains that fear reactions are incapable of complete ex 
tinction and once learned are capable of motivating behavior forever 
Some avoidance reactions, however, extinguish rather easily Only those 
developed on the basis of traumatic unconditioned stimuli seem com 
pletely resistant to extinction Tor this reason, Solomon and Wynne 
suggest, as Hull had earlier, that the intensity of the UCS is the impor- 
tant determiner of this irreversibility 

SUMMARY 

Contemporary views on the nature of reinforcement Und to be of more 
limited scope thin the general theories reviewed m the last chapter The 
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most important of these newer positions identify reinforcement with the 
reduction of tension, the evocation of a response, or the presentation of a 
stimulus. 


Tension-Reduction Theory 

There have been three important versions of the tension-reduction posi- 
tion. (1) The earliest of these was the theory of need reduction which 
held that reinforcement always entailed the satisfaction of a basic bio- 
logical need. This theory had to be abandoned because of numerous ex- 
ceptions to the general principle it proposed. The phenomenon of sec- 
ondary reinforcement, as well as the facts that the onset of a light and the 
taste of saccharine are reinforcing are difficult to incorporate in such a 
theory. (2) The drive-reduction theory treats motives abstractly and re- 
lates them to appropriate anteeendent conditions. Although many of the 
drive-establishing operations employed in the definition of drives are the 
same as those which produce states of biological need, the distinction 
between drive and need is possible in drive-reduction theory. (3) Finally 
there is drive-stimulus-reduetion theory. In this view drives are regarded 
as intense stimuli and the reduction of this stimulation is considered to 
be reinforcing. 


Alternatives to Tension Reduction 

Tension-reduction theory has so dominated thinking about the 
nature of reinforcement that it is widely accepted as THE reinforcement 
theory, and alternative positions are incorrectly regarded as non-rein- 
forcement theories. These alternative views of the nature of reinforce- 
ment are: (1) a response theory which holds that certain forms of be- 
avior, most of them consummatory behavior, are reinforcing and (2) a 
s lmu ational theory which maintains that reinforcement reduces to a 
a er of stimulation. These positions have not been developed to the 

as tension-reduction theory, being most applicable to certain 
experimental situations. 


Physiological Studies of Need Reduction 

dtmttorZvett^ f ^ t0 “k^ 11 data rde ™>‘ ‘he ™l» of need re- 
driveor need j , q “' X”* wUch are essentially opposite: (1) Is 
in"! (O) Xs it Tin C h? n . a sadaC . lent eondition for the production of learn- 
no'drives or neX ° btam ““forcers which reduce 

JduXXrXvTf I,"" ‘ ie ** *“■«“ *"• shown that food in- 
oclueed directly into the stomach and glucose (which raises blood 
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sugar) injected into the blood stream are adequate reinforcers These 
results support a need reduction interpretation of reinforcement On the 
other hand experiments prompted by the second question have shown that 
saccharine, exploration, and electrical stimulation of the brain can also 
function as remforcers In none of these cases is there the reduction of 
any obvious drive or need Such data, therefore, favor an interpretation 
in terms of stimulational effects 


Brain Stimulation and the Stimulus Theory 
Of the results just mentioned the one of particular interest is that a 
mild electrical shock applied to certain areas of the brain (limbic system) 
through chronically implanted electrodes is capable of reinforcing many 
kinds of learning This raises questions of interest to the neuroanatomist 
and neurophysiologist as well as to the psychologist Experiments ha\e 
shown that the effect of stimulating different areas of the brain -varies 
and that, with some electrode placements, brain shock is a negative rein 
forcer Whether a bram shock is a positive or a negative reinforcer also 
depends to a degree upon the intensity and duration of the electrical 
stimulus and upon the drive state of the animal This last result provides 
the very strong suggestion that brain stimulation is related to the process 
of reinforcement as it normally occurs in conventional experiments on 
learning At the same time, it must be recognized that the details of this 
relationship are not well worked out Much additional research needs to 
be done on the operation of the usual remforcers, as well as upon bram 
stimulation 


Response Theories of Reinforcement 
Theories which treat reinforcement in terms of responses have never 
been very popular, possibly because of methodological difficulties winch 
stand in the way of clearly separating them from stimulational theorj 
and, in some cases, from drive reduction There is, however, evidence 
which leads in the direction of such a theory It has been shown that 
there is a very high correlation between the vigor of consummatory be 
havior and the vigor of the instrumental response in certain learning 
situations There is also evidence that activity and manipulation will 
serve to reinforce learned acts 


Two-Process Theories 

Avoidance learning creates problems for any theory of reinforcement 
Since a successful avoidant response prevent ,} ie occurrence of t i<? 
rcinforcer, it is difficult to understand how such beliav lor could he mam- 
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tained. Yet it is known that such responses can be highly resistant to 
extinction. Several writers have developed two-process theories to explain 
the characteristics of avoidant behavior. These theories assume that, 
early in the course of avoidance learning, a process of classical condition- 
ing occurs in which a fear response, initially elicited by the noxious UCS, 
becomes conditioned to the CS. This acquisition of fear is assumed to 
take place without drive- or need-reduction. Once acquired, the condi- 
tioned fear is thought of as a drive, capable of motivating behavior. 
Escape from the fear-evoking CS is reinforcing through the mechanism of 
drive reduction. 

This theory, though attractive, raises certain questions: 

1. Is fear conditioning, as hypothesized, a case of learning by conti- 
guity alone? Studies in which the duration and rate of cessation of the 
UCS have been manipulated in classical conditioning offer an affirmative 
answer. Since a UCS of long duration theoretically delays drive reduc- 
tion and gradual cessation of the UCS means that drive reduction occurs 
gradually, learning, according to drive-reduction theory, should occur 
slowly under these conditions. It has been found, however, that neither 
of these variables has any effect. The most obvious predictions from 
drive-reduction theory, thus, turn out to be in error. Studies of the in- 
tensity of the UCS have demonstrated quite convincingly that this vari- 
able influences learning. But they have contributed little or nothing to 
the question whether classical conditioning is a process of contiguity 
learning. 

2. Why is avoidant behavior so persistent? Since the avoidant re- 


sponse prevents the occurrence of the UCS, it might be expected that 
extinction would set in as soon as the response became at all consistent. 
In many experiments this does not happen. In order to explain the great 
resistance of avoidance behavior to experimental extinction, two addi- 
tional principles have been proposed: (a) the principle of anxiety con- 
servation, which is that fear is so resistant to extinction because the 
avoidant response occurs before fear can be aroused, thus protecting the 

' a ” d <6> ^ PrineiP ’ 6 ° f Partial “'-'-ersibility, 
svh ch is that fear is inextinguishable if it is based upon a very intense 
and traumatieally painful stimulus. 1 7 
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the CS is repeatedly presented unaccompanied by the usual rein 
forcer the conditioned response undergoes a progressive decrement called 
extinction and finally fails to occur Tins reduction in the strength of 
the CR is not a spontaneous decay Conditioned responses show only a 
very limited tendency to dimmish with the passage of time Studies of 
the retention of CR s haxe usually shown some decrease m strength with 
disuse but a much more impressive retention) The numerous demonstra 
tions of the great resistance to CR s to forgetting include reports of the 
retention of conditioned motor responses by sheep for two jears (Liddell, 
W T James and 0 D Anderson 1934) conditioned eyelid reactions 
m dogs for 16 months (D Q Marquis and Hilgard 1936) conditioned 
eyelid reactions m man for 20 weeks (Hilgard and A A Campbell 1936) 
and for 19 months (Hilgard and Humphreys, 1938b) , conditioned flexion 
reflexes in the dog for 30 months (Wendt, 1937) , conditioned salivation 
m man for 16 weeks (Razran 1939a) \arious responses in dogs for 6 
months (Kellogg and Wolf 1939), and a pecking response m pigeons 
for four years (Skinner, 1950)\From such evidence it is apparent that 
the elimination of a conditioned response in a brief extinction session 
must depend upon an active process or perhaps several of them 

The determination of the basic mechanisms which underlj extinction 
is difficult because of the great complexity of the phenomenon We bate 
previously seen that resistance to extinction is a measure of learning a 
fact which means that to understand extinction it will be necessary first 
of all to know how the conditions of training affect habits To mention 
facts which appear to be of greatest significance for extinction the 
omission of rein f orcein ent alwaxs lias at least sonic of the following 
consequences (1) inhibition or adaptation of the response mechanism 
(2) the instigation of interfering responses^ (3) g cnerjlw aUOlUilecrP 
ment, r esulting from changes in the stimulus situation (4) dccnasu m 
the l ex el of m< fixation which is most obxious when extinction mxolxcj 
the omission of a negatixe rcinforcer and (o) frustration, winch ix most 
obxious when extinction inxolxcs the omission of i positixe rcmforccr 
Various writers base stressed different aspects of extinction, each tending 
2S1 
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to favor one or two of the processes just listed. The result has been the 
development of a series of hypotheses to account for extinction. The most 
important of these hypotheses are described in the following sections. 


INHIBITION THEORY 


;_The central concepts in early theories of extinction were those of inhibi- 
tion and interference, both of which were drawn from physiological v T ork 
on simple reflexes (Sherrington, 1906 ) and from Pavlov’s studies of 
conditioning. In these original contexts both concepts w 7 ere employed. 
When they were taken over by psychologists, however, there was a strong 
tendency for one concept or the other to be emphasized. In this way two 
alternative theories of extinction gradually developed. One adopted the 
principle of inhibition and attempted to explain all extinction phenomena 
in terms of it. The other attempted to do the same thing using only the 
principle of interference (Guthrie, 1935 ; Wendt, 1936 ). 

The most nearly contemporary version of classical inhibition theory is 
to be found in a j)art of Hull’s ( 1943 ) explanation of extinction. In this 
theory, Hull assumed that every response of the organism, w r hether rein- 
forced or not, left in the organism an increment of reactive inhibition 
(I R ), the magnitude of which was an increasing function of the rate of 
response elicitation and the effortfulness of the response. It was assumed 
to decay with rest. Hull further assumed that reactive inhibition is a 
~^^^^^^A^BZg_IgsgMblin g fatigue w hich leads to a cessation of 
^gg£gMlZM,PAoducesjt. Formally, reaetivelnMbition w^a p^7- 
formance variable m Hull’s theory, subtracting from the combined effect 

strength^ 0 m °tivation (sEh) and, as a result, lowering response 


Before considering this theory as an explanation of extinction, certain 
more general observations need to be made about the concept of inhibi- 

even iuT 6 ^^ically develops with responding 

ZcernMein ? Tei f 0TCed ’ the eff ects of this variable will be 

applTcatbn of perfo ™ ance ^nations. More or less successful 

'“Tv b6en made t0 Vernation behavior 

“n of practice effects 

tisi o f h :r eption of 
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level of is ,l,at mMbition reduces the 

ornanee. Thus any conditions which increase the amount of 
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inhibition should have a detrimental effect upon response strength The 
two most important of such conditions are the distribution of extincti on 
jj‘ ia l s an d the effortfulness of the response A s regards the first of these 
variables, the specific prediction to which inhibition theory leads is that 
extinction will occur more rapidly with massed than with distribut ed 
JnalsjThe results of several of the most straight forward tests of tins 
prediction are summarized in Table 20 By “straight forward” we mean 
that interest m these studies has centered exclusively, or nearly so, on 
the effect of the distribution of trials, while other factors such as 
generalization decrement, have been left out of consideration As Table 


TABLE 20 

Results of Studies of Effect of Intertrial 


Interval on the Rate of Extinction 


Investigator 

Response and 
Subject 

Condition* of 
Distribution 

Results 

Pavlov (1927, p 53) Salivation, 
dog 

2, 4, 8, 16 mm 

No effect on number ot 
non reinforced trials 

Hilgard and Mar 

Eyelid CR 

60 trials per day 

■ Faster extinction with 

quia (1935) 

dog 

versus 10 trials massed trials 
every other 

Porter (1938c) 

Running 20 
foot alley, 
rat 

A few seconds, 

5 minutes, 

10 minutes 

No effect 

Porter (1939) 

Eyelid CR, 
human 
subjects 

10, 20, 40, 80, 
180 second* 

No effect 

Gagnd (1941b) 

Running 

5, 1, 2, 3, 5, 10 

Interpretation complicated 

(Exp 1 only Exp 

3 foot ele 

minutes, in 

by differences due to acqui 

II complicated by 

vated maze, 

acquisition 

sition distribution ami in 

different amounts 
of reinforcement) 

rat 

and extinction 

bibition of reinforcement. 
Probably no effect. 

Rohrer (1947) 

Bar pressing, 
rat 

10, 90 seconds 

In general, faster cxtinc 
tion with massed practice 
Suggests increased effect 
with stronger responses. 

Rohrcr (1949) 

Bar pressing, 
rat 

10, 90 seconds 

Faster and more perma 
nent extinction with nassod 
trials 


Running 

1, 5, or 15 trials 

5 and 15 trial groups 

A nisei (1954) 

C foot alley 
to cscapo 
shock, rat 

a day 

showed extinction tendency 
Little difference between 
theso 2 groups. One trial 
group showed little cxtinc 
tion 


20 shows the most common result lias been for massed jiricticc to s pec 1 
‘Up ext inction There have however, been enough experiments m which 
thufcffect failed to occur to suggest that manipulating the distribution 
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variable must also produce changes in processes other tha n inhibition^ 
The most likely additional process has to do with a relationship between 
the distribution of practice and the nature of the stimulus situation, a 
topic to be taken up later. 


The Phenomenon of Spontaneous Recovery 

^Somewhat more substantial support for an inhibitory concept comes 
from studies of spontaneous recovery, although again the details of the 
evidence soon demonstrate that extinction must involve additional fac- 
tors. It will he recalled that the phenomenon of spontaneous recovery i s 
that, following extinction, rest produces an automatic increase in re - 
s pon s e „str eu gtliT The most obvious explanation for this increase is that 
some temporary inhibitory state, produced by non-reinforcement, decays 
during the rest interval. There are, however, many difficulties in the 
way of explaining everything that is known about spontaneous recovery 
m terms of an inhibitory process. Some of the problems are these: (1) 

. .. , < ^ response strength back to the level 

it had attam^ prjo^io_c pnditioning . Pavlov statprl that 
selves, extinguished reflexes sooner or later returned to their full 
strength, and he presents evidence to support his claim (Pavlov, 1927, 
p. o8-o9) ; but this finding seems to be the exception. More often spon- 
taneons recovery restores something like 50 per cent of the response 
strength lost m extinction. Prom the inhibitory point of view this creates 

a J™ 1 * 7 ’ because *° Permanent negative process has been assumed. 
(-) e assumption that spontaneous recovery is the result of the dissipa- 

lea ? i° the g6neral P r ^etionJhat_pr pcedures which 
increa s e the amou nt 

W on D J [ C ° nCe ™ir th S6Veral "P^ 5 ° f action and ex- 
SafXtaL ; / T a i 9 f- ) estm ^ed the response of running a 
trials He fou^rl °° d SeCOnds or two minutes between extinction 

~ous recovery was 
CS after the f’P 1,0=. , . , s ; Rep ea ted presentations of th e 

zero” is hard tr, ,i“: • T — 1 — — lhis extinction “bel ow 

There^Me^om^jmzzW unexMainlf *'“P onse 4>roduced inhibition. (4) 
taneous reoovp gigg£gnees among curves of spon- 

Tiocess^ SomTtWlhT^Tl^J^^ 

few minutes after extinction (Pavlov 1997 & 

1940) ; but in Other experiments it bJ ~ ’J' 8 ’ Graham Gagne, 
for an hour or several hom-c i Vu S sometmes been necessary to wait 
to occur (Ellson, 1938; D. J. Lewis ^ cpfi f 6 ?" 66 of s P ont aneous recovery 

5 0 ^ seems improbable that the 
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extended spontaneous recovery functions result from the same process 
as those of shorter duration^} 

Extinction and the Effortfulness of the Response 

S 'rom the assump tion th at the amou nt of reactive inhibition varies with 
effort required to perform a response, it follows that increased eftort- 
T uiness will result in quicker extinction of the response in questio n 
Although Maatscli, Adelman, and Denny (1954) obtained negative re 
shits, experiments testing this prediction have, more often than not, co n- 
jrmed the prediction ( Mowrer and Jones, 1943, Solomon, 1948, Apple- 
zweig7 19&1 , StanleyTnd Aamodt, 1954, Capehart, Viaey, and lluhcka, 
1958) The experiment of Capehart, Viney, and Hulieha illustrates the 
typical procedure employed in such investigations Rats were trained to 
press a bar in a Skinner box to obtain food In order to balance out 


experience with the various degrees of effortfulness later to be compared, 
all animals received 30 trials with bar weightings of 5, 40, or 70 gm 
during the initial training Then they were divided into three groups 
and extinguished under conditions where it took different amounts of 


pressure (5, 40, or 70 gm ) to press the bar The results of this study 
appear m Figure 10 1 Quite c lear ly, i ncreasing effort leads to inercu s 
jngly rapid extinction 

^Although the evidence seems clear on the point that effort does influ 
enee extinction, the form of the function, and the importance of this 
variable, are still in doubt This is because the problems of manipulating 
effort, but no other variable^m studies of extinction are greater than 
might have been anticipated / Tn the majority of m\ estigations, which 
have used a Skinner apparatus and produced different degrees of effort 
by differential bar weightings, two procedural difficulties arise (1) In 
order to train the animal to press a heavy bar it is usually necessity to 
begin by teaching it to press a lighter bar first Such training mi} , in 
^effect, give the animals extinguished on the light bar more training than 
those extinguished ex a heai .w one (2) The preMe.vi oS partial resptau >r> 
is sometimes critical Such responses, which consist of bir depressions of 
insufficient magnitude to operate the food release mech mism, occur 
more frequently with a heavy bir than with a light one During training, 
the occurrence of these responses introduces uncontrolled schedule effects, 
and during extinction, one must decide whether to count them If they 
are counted, the difference between groups extinguished with responses 
requiring different amounts of effort necessarily decreases (Applczweig, 
1951, Stanley and Aamodt, 1954) Other ways of mampuhting effort 
produce other difficulties Solomon (J948), for example, mere iscd and 

'For further studies and <])«-usjjou of spontaneous reco\rry «*o Utr» 0313), 
Liberman, (1014, 1043), Murphy, Miller, and Imnixhio (lO'O), Millir *nd Mile* 
(1030), ilillcr aud btoeusou (103i»), Itarran (1030b), and louU (lOJ^a) 



284 


CONDITIONING AND LEARNING 


variable must also produce changes in processes other tha n inhibition^ 
The most likely additional process has to do with a relationship between 
the distribution of practice and the nature of the stimulus situation, a 
topic to be taken up later. 


r 


The Phenomenon of Spontaneous Recovery, 

^Somewhat more substantial support for an inhibitory concept comes 
from studies of spontaneous recovery, although again the details of the 
evidence soon demonstrate that extinction must involve additional fac- 
tors. It Avill be recalled that the ph enomenon of spontaneous recovery is 
th at, following extinction, rest produces an automatic increase in re - 
spons'e_sliimgth7Tlie most obvious explanation for this increase is that 
some temporary inhibitory state, produced by non-reinforcement, decays 
during the rest interval. There are, however, many difficulties in the 
yay of explaining everything that is known about spontaneous recovery 
in terms of an inhibitory process. Some of the problems are these: (1) 
Spontaneous recovery seldom brings response strength back to the level 
it had attaine d^pnor_tolcpnhitionmg. Pavlov stated tha^TeftToTEem- 
selves, extinguished reflexes sooner or later returned to their full 
stre “ gt J’ and lie P res ents evidence to support his claim (Pavlov, 1927, 
P- 58-o9) ; but this finding seems to be the exception. More often spon- 
taneous recovery restores something like 50 per cent of the response 
n r< rm 'i f ° S , m Emotion. From the inhibitory point of view this creates 
t^ U eC f. USe P ermane nt negative process has been assumed. 
Li nf 6 L at spoutaneous recovery is the result of the dissipa- 

increaqp 1 ? Lv* 1 ^n 8 i° £ enera l predictio n that procedures which 

incre a se the amount o f^L^^^ 

trials ^' Condition wffichmi ght bring th is 

ttct on D J I 7 COnC n n Q Si With SeVeral aspects of a *^tion and ex- 
stra Xt’aLL L L eXtin S uisbed the response of running a 
triaL He finn l ^ m SeC ° ndS ° r tW ° ™ tes between extinction 
ncreasfd bt matT f eVidenC6 th&t s P°^aneous recovery was 

(3) fetej Presentations of th e 

zero” is hard tn ^ — syun[ i _iy^7_). This extinction “belo w 

Xhe re are son,, m .,ii inhibition. (4) 

tanprmo yan ™* — _L iTr 5 ^*2 d_differences among* curves of spon- 


for an honi oV^Cr^'thef T*” b “" T *» -H 

to occur (Ellson, 1938; D J Lewis iLfi of spontaneous recovery 

• ewis, I95fa). It seems improbable that the 
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extended spontaneous recovery functions result from the same process 
as those of shorter duration 

Extinction and the Effortfulness of the Resp onse 

e Vom Jhe assumptio n t hat the amount of reactive inhibition v anes with 
effort required to perform aTrespon.se, it follows that increased effort- 
fulness will result in quicker extinction of the response in questio n 
Although JVIaatsch, Adelman, and Denny (1951) obtained negative re- 
sults? experiments testing this prediction have, more often than not co n- 
firmed the prediction ( Mowrer and Jones, 1913, Solomon, 1918, Apple 
zweig, 1951, Stanley lind Aamodt, 1954, Capehart, Yiney, and Iluhcka, 
1958) The experiment of Capehart, Viney, and Hulicka illustrates the 
typical procedure employed in such investigations Rats were trained to 
press a bar in a Skinner box to obtain food In order to balance out 


experience with the various degrees of effortfulness later to be compared, 
all animals received 30 trials with bar weightings of 5, 40, or 70 gm 
during the initial training Then they were divided into three groups 
and extinguished under conditions where it took different amounts of 


pressure (5, 40, or 70 gm ) to press the bar The results of this study 
appear m Figure 10 1 Quite cle arly, i ncreasing effort leads to mcrcas 

seems clear on the point that effort does influ- 
ence extinction, the form of the function, and the importance of tins 
variable, are still in doubt This is because the problems of manipulating 
effort, but no other variable, in studies of extinction are greater than 
might have been anticipated/ In the majority of investigations, which 
have used a Skinner apparatus and produced different degrees of effort 
by differential bar weightings, two procedural difficulties arise (1) In 
order to train the animal to press a heavy bar it is usually necessary to 
begin by teaching it to press a lighter bar first Such training nny, in 
'effect, give the animals extinguished on the light bar more training thin 
those evtiagui'sited Cfi a keai tec esse (2) The prehiem eS fXtrl.\s! respe.'-'^ef 
is sometnnes critical Such responses, which consist of bar depressions of 
insufficient magnitude to operate the food release mechanism, occur 
more frequently with a heavy bar than with a light one During training, 
the occurrence of these responses introduces uncontrolled schedule effects, 
and during extinction, one must decide whether to count them If they 
are counted, the difference between groups extinguished with responses 
requiring different amounts of effort necessarily decreases (Applczweig, 
1951, Stanley and Aamodt, 1954) Other ways of manipulating effort 
produce other difficulties Solomon (1918), for example, inert i-cit an ji 

‘for further studies and discussion of spontaneous rrcotrry sec h»fe« 

Liberman, <3.114, l n 4S), Morphy, -MiJI*** 1 , and tinnoehio (U*So), Milt.r and ililr* 
(103d), Miller and fattunson (1J3V), ltaxran (1J3M»), and louU (li>3«a) 


mgly rapid extinction 
^Although the evidence 
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FIGURE 10.1. 


The effect of effort upon resistance to extinction (Capehart, 
Vmey, and Hulicka, 1958). 


stand-in tV 1 ^ ^ ^ diStanCe ratS h&d t0 W in a 0 um P in § 

well inorP^ t^ 1 ’ 06 ! Vc nC1 ' eaSing length of tlle ^mp “ay very 

well increase the animals’ fear. J 


The Problem of Partial and Varied Reinforcement 

the effect of SialreLwtS fo^ VR eStinCti ° n is to esplain 
ties are particularly great. AltkoimAf- lnhl ^ lt f n theor y, the difficul-. 
the effects of nar+ini • $ ° dinner bad previously described 

«bra^ a S feeS:tTarbl UPOn «“ ***■»• » f 

"’ort._In_ 1939 , Humphreys ^oognKed opiy after Ilmnphreys’ 

force(Tl^on-rpiuw I^^|^==^- random alternation of rem- 
jowering of the^ tf^ft^A;^ } &d t0 & sligM 

Reinforcement schedule and +n 7 comparison with a continuous 

schedule, and to av ery much greater registame to egtinc- 
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_tion_ 2 To explain this finding Humphrey's offered an interpretation m 
J grms of expectancy (Humphreys, 1939a, 1939b) 

The impressive thing about the “Humphreys' effe ct” or “ .Humphreys' 
jparadox " to investigators in the early forties was that it seemed to 
threaten two widely accepted principle^ the law of effect and th e mhih i 
tory explanation of extinction With respect to the first of these, the 
point was that an expected correlation between num bers of re inforce • 
ments and r esponse str ength (measured in resistance to extinction) failed 
to materialize With respect to the second, the problem was to explain 
the fact that nearl y identical levels of learning r equired for their extinc- 
tion ^yastly_different_numbers__of_^esponses_and, therefore, different 
- amounts of inhibitio n 

[^Studies of what has come to be calle d varied reinforcement ( McGlplInnrl 
an<l McGowan, 1953) are related to studies of partial reinforcement in 
that they are concerned with learning under circumstances where the 
conditions of reward ehange from trial to trial But instead of omitting 
reward on certain trials, the procedure in these studies is to change other 
aspects of reinforcement according to some schedule The possibilities 
are quite numerous, including the place where reinforcement is admin 
istered from trial to trial, the delay of reinforcement, the quantity or 
quality of reinforcement, the drive state of the animal, aud the specific 
response leading to reward Ordinary continuous and partial reinforce 
meat conditions are sometimes included as control conditxons^Iachintosh 
(1955) ran 12 different groups of subjects, varying three different 
sources of irregularity systematically among them The sources of irregu 
lanty were (1) Percentage of reinforcement, 80 or 100 per cent (2) 
Number of response alternatives, 1, 2, or 3 The response possibilities 
were pulling a chain, pressing a horizontal bar, and pressing a vertical 
bar For the different groups the correct response was alwajs the same 
(chain pull), one of two responses (chain pull or horizontal bar press), 
or one of three responses (3) Number of reinforcers employed (one or 
two) Under one schedule the reinforcement was always food , on another 
it was food ou some trials and water on others The subjects were always 
both hungry and thirsty The apparatus was that shown in Figure 102 
The problem for the animal was to operate the correct mechanism , winch 
opened a door allowing access to the reinforcement The various rem 
foreement conditions combined in all possible ways yield a dimension of 
irregularity of reinforcement extending from 100 per cent reinforcement 
— -one response— one reinforcement, to 80 per cent reinforcement— three 
response alternatives varied from trial to trial— two rewards varied from 
trial to trial In general, number of trials to a strict extinction criterion 


» tor a variety of demonstration, of Uni point wo Jcnktn. and Rifiby. 1**0 . JenUns, 
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FIGURE 10.2. Apparatus employed in a study of the effects of irregular rein* 
forcement upon extinction. See text for a description of the apparatus (iYIac- 
Kintosh, 1955). 


increased 'with increases in irregularity, from about 45 trials, for the 
most regular condition, to something like 200 for the most irregular. 
Analysis of the contribution of the various sources of irregularity showed 
that the type of reinforcer did not contribute to the increased resistance 
to extinction. Schedule of reinforcement and number of response possi- 
bles produced the entire effect. The fact that varying appropriate 
reinforcers does not increase resistance to extinction has subsequently 
been confirmed (Kendler, Pliskoff, D ’Amato, and Katz, 1957). 

nother experiment in this same tradition extends the range of condi- 

nTsfrT + t0 • affeCt extinction ' In this study McNamara and Wike 
(19o8) first trained o different groups of rats to run a straight alley. 

ne, a controi group, was trained under standard conditions, involving 
ofiwLrr™ 10 ! m the eXperlmental situation or in the conditions 

conditions tS p^eT ^ * lining 

chantm^ «-nr.v, 1+ . t ) by making a series of environmental 

box aad ® on (S evel 0f changing the goal 

some tnab and 20 1 l" 3 ™ 8 ‘‘‘ »f reward, 0 seconds on 

product and (4 T °" ° (3) ^ combi ™g ^ too previous 

o£ reinforcement, drive a nd 'rew 'll 7 °* "sMorcement, percentage 
simultaneouslv Title 1 + reward, and environmental condition 

lXTfto 36 trii f " re WaS “ attem P‘ irregn- 

guished. S “^“tt 5““’ a ™ Is ^ 
running time in Pia Ure 10 3 0h • “ ° UpS are P res ented m terms of 

the training conditions increases restoneeTo' exSon, ° f 
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FIGURE 10 3 Median running time in seconds for groups trained under a 
variety of conditions of reinforcement In the control group there was minimal 
variation in the conditions from trial to trial The sources of \anation from 
trial to trial are given in the legend Cue and response variation was accom 
plished hy inserting hurdles in the runway, changing illumination, and shift 
ing goal boxes Variations in delay of reinforcement were produced hy admin 
istenng reward immediately on some trials and after a delay of 20 seconds on 
others To manipulate drive, animals were hungry on some duys, thirsty on 
others, and both hungry and thirsty on still others. Reward was dry food 
pellets on hunger days, water on thirst da>s, and wet nush on hunger thirst 
days Note that, in general, the greater the number of sources of variation, 
the greater the resistance to extinction After MeN amara and WiLe {19531 

Other investigators have studied the effect of varied reinforcement in 
a more analytic vvaj, bv manipulating the ngnlantj of some single 
lspect of reinforcement {magnitude, dcJav pattern) Crum, Hroun and 
liittirmnn (1971) have reported the results of two experiment* which 
show that varied temporal intervals of tuo Mirts product* men istij re 
instance to extinction In the first of their experiments fear was ccmdi- 
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tioned to a 6-seeond buzzer under three different conditions. In one 
(control) condition, a 2-second shock came on at the cessation of the 
buzzer. In a second (partial reinforcement) condition, the shock was 
omitted on 50 per cent of the trials. In the last condition, shock occurred 
on every trial but was delayed 30 seconds following the cessation of the 
buzzer on half of the trials. Following this conditioning, all of the sub- 
jects learned to escape a shock by running from one section of a two- 
compartment bos to another. Finally the shock was omitted and the 
buzzer reintroduced. On these trials the buzzer was at first capable of 
eliciting the escape reaction. This tendency disappeared (extinguished) 
in 12 trials for the control group, 95 for the partial reinforcement group 
and 107 for the varied delay group. The implication is that varying the 
interstimulus interval from trial to trial has at least as strong an effect 
as a partial reinforcement condition. 


In their second experiment, Crum, Brown, and Bitterman used food 

reinforcement and a straight runway, a procedure which has been 

employed in much later work. One of two groups received immediate 

reinforcement on each trial; the other received immediate reinforcement 

on half of the trials and reinforcement after a 30-second delay on the 

other. The first of these groups required 13 trials to extinguish; the 

second required 31, again showing that varied reinforcement produces 

greater resistance to extinction than regular continuous reinforcement. 

Scott and Wike (1956) repeated the Crum, Brown, and Bitterman 

experiment and obtained the same results. Peterson (1956) and Logan, 

Beier, and Kmeaid (1956) have also found that extinction of a runway 

response is retarded when training is under a variable delay schedule. 

• , /“ er lnVe ' Stlgators Iound sucd an effect only when training con- 

Snn°nVV u of °’ ^d 30-seeond delays. Similar alter- 

nation ox 0- and 9-second delays had no such effect. 

following KillCaid 8150 com P are d resistance to extinction 

Jw constant magnitudes of reinforcement. SpecifleaUy 
on eveT S 1 * <D » pellets of food 

reinforcement} or A} Q 6 itT S ° me an '^ nfme 011 others (partial 

’ To ( l T “ S ° me triab ® others (varied 

«iai the varied md p %- 

Gtl °™ ’*■» appropriate eompTiaons can 

^hel^Tquestion as the heS6 mvestlgatl0ns raise 

strengths are similar at the end of . rC . eme “ ± studleS: Since respon se 
eAplaxh the tact that they are so 

tneyar^ S o different in resistance to extinction? 
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The Response-Unit Hypothesis 

In the face of the facts of partial and varied reinforcement, most 
theorists would probably agree that inhibition theory must be supple- 
mented by other principles if the details of extinction are to be accounted 
for adequately. There is, however, one interpretation which offers some 
slight possibility of dealing at least with the partial reinforcement data 
in terms of inhibition. This is the response-unit hypothesis. Mowrer and 
Jones (1945) performed an experiment in which rats first receded 80 
continuous reinforcements for bar-pressing. Following this, one group 
remained on the continuous schedule; three other groups were switched 
to fixed-ratio schedules in which they had to press the bar 2, 3, or 4 times 
to receive the reinforcement; and a final group was switched to a variable- 
ratio schedule in which reinforcement occurred, on the average, after 
2 5 responses. All groups received 140 reinforcements on the new sched- 
ule. Then they were extinguished, with the results shown in Figure 10.4. 



FIGURE 10.4. Resistance to extinction as a function of ratio of reinforced to 
non reinforced responses. In the upper curve, responses, are plotted direcljy. 
Lower (response unit) curse was obtained by multiplying the values in the 
upper curve by tile reinforcement ratio. Points not included in the curve 
itself are for the randomly reinforced group- See text. Mowrer oud Jones 
(1945). 

As the upper curve in Figure 10 4 shows, the usual effect of partial re- 
inforcement occurred. With increasing ratios, the number of responses 
emitted in three dajs of extinction increased Mowrer and Jones, how- 
ever, orgued that this tread appeared only because each bar depression 
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•was counted as a response. If the concept of response were redefined so 
that a response becomes a behavioral sequence leading to reinforcement, 
it would be necessary to multiply the number of extinction bar depres- 
sions by the reinforcement ratio (1/1, 1/2, 1/2.5, and so on) in order 
to obtain the true number of responses. The effect of counting responses 
in this way is shown in the lower line of Figure 10.4. It will be noted 
that the trend in the results is now the opposite of that obtained by 
counting each bar depression as a response. Moreover, the downward 
slope of this function fits the necessary prediction from inhibition theory, 
that more effortful responses (consisting of more bar depressions) should 
extinguish more rapidly than less effortful responses. Thus, in this case, 
the response-uxnt hypothesis salvages the inhibition theory. It seems un- 
llkely, however, that this explanation will always work, for two reasons: 
(1) The response-unit hypothesis does not seem to apply very well to 
trial-ly -trial procedures, particularly if the trials are widely spaced. It 
is difficult to imagine the sense in which two successive responses sepa- 
rated by 24 hours, as occur in some experiments on partial reinforcement, 
constitute a unit (2) Certain facts about extinction argue against it. For, 
example, Capaldi (1958) gave rats 10 trials a day in an apparatus which 
consisted m an elevated runway terminating in a jump stand. To obtain 
ood the subject had to run down the alley and jump across a gap. 
apa ran, among others, two groups of rats which received reinforce- 
ment or alternate trials. One group received 70 trials; the other 140. In 
terms of response-units, as defined by Mowrer and Jones, the first of 
these groups received 35 trials and the second, 70. This should produce 
greater response strength and greater resistance to extinction in the 

animaW^ViJ 116 ° Pp0Slte ’ however > turned out to be the ease. The 

reach^n^ v SmaU + er . amoilIlt of training required about 67 trials to 
reaeh an extinction criterion ; the group with more training required onlv 

tts?To ** ^ t6rmS ° f the discrimination hypo- 

thesis to be discussed later. The group with more training he argued 

becaif „?thdr mrT 6 ‘™ erenCe iehre “ aad extinction 

procedure. This is a'sort of ^2“"' 
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O n. the negative side, against the view that inhibition provides a com 
plete explanation for extinction are the rather numerous instances where 
the_d istnbution p£ extinction trials fails to have the predicted effect , the 
fact th at_spontaneous recovery is seldom able to restore an extinguished 
response c ompletel&Jhe pheno menon of extinction below zero and the 
impossibility of handling the influence o£ partial sad varied reinforce 
ment upon extinction e xcept in a limited number of ensps ’ ~~ 

The negative evidence just cited indicates the necessity for supple 
menting inhibition theory with other principles in order to explain a 
greater number of the facts of extinction One source of such additional 
principles obviously is the rest of the theory which attempts to explain 
extinction in terms of inhibition _ Hull's theory, which is the clearest 
case in point contains numerous possibilities of this sort According to 
this theory the extinction procedure might lead to a decrease in response 
strength through a reduction of incentive motivation or second try rein 
_forcement, t hrough th e reduction of habit strength w ith changes in the 
amount and delay of reinforcement (Dufort and Kimble 1956), or 
Trough “geneiaiization decrement resulting from stimulus changes cor 

related with the shift from conditioning to extinction Of these various 
possibilities the last has been the only one to have received much serious 
consideration We shall turn to it next 

GENERALIZATION DECREMENT 

All extinction procedures involve changes m the experimental situation 
in that the proprioceptive consequence of reinforcemen t and eventual/* 
responding are eliminated If the conditioned response is at all under 
the control of these stimuli it should lose strength is a result of suc h 
changes, and extinction should be hastened to a degree which depend s 
upon the magnitude of jthesc differences in stimulation between condi 
turning and extinction The generalization decrement hypothesis stresses 
this interpretation 

As was true of the concept of inhibition generalization decrement his 
applications beyond the field of extinction The general principle that 
changes m the learning situation result in response decrements has been 
verified in a variety of ways One obvious example is in the phenomeno n 
of external inhibition It has also been shown that the drinking behavior 
of rats diminishes progressively ns more and more of the customnrv 
environmental cues to drinking arc eliminated (I mb and ration 1J»J) 

And the warm up phenomenon characteristic of the performance of 
motor skills (Ammons, 1JI7) and rote learning (Thunc UoO) sums to 
be a matter of overcoming a dicmncnt produced by tin loss with rud 
o f resgogsc produced cue* w Inch control the learned response m seme 


measure 
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Generalization Decrement and the Distribution of Practice 

This last example has important implications for extinction, especially 
when related to the concept of inhibition. Inhibition, as it is usually con- 
ceived, is a response process. Hull’s term reactive inhibition suggests an 
effector-localized mechanism, as does his treatment of I u as something 
analogous to fatigue. If this interpretation is accepted, it is important to 
recognize that responses, at least skeletal ones, almost always have stimu- 
lus consequences in the form of kinesthetic feedback (Solomon, 1948), 
and that these stimuli may play a very important part in the control of 
behavior. 

The generalization-decrement hypothesis, together with the concept of 
response-produced stimuli, suggests a more detailed analysis of the effect 
upon extinction of the distribution of extinction trials. This analysis 
applies most directly to experiments which alter the conditions of distri- 
bution at the beginning of the extinction series. Responding, under a 
given distributional condition, would be expected to maintain kinesthetic 
feedback at some particular level. Changing the degree of distribution 
for extinction, or for further practice, would lead to a change in the 
level of response-produced stimulation and, therefore, to a loss of re- 
sponse strength through generalization decrement. 

Experimental evidence in general support of such a theory comes 
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JlSLrr 1 Iu 0n,! of fte earHest deoonslraHoas (Seynolds, 

1945b), the eyelid reflexes of groups of iumau subjects were ctmditioued 
mlh aa tutertnal interval of either 15 or 90 seconds. Then, for esthictim 
these main groups were subdivided, half continuing under the original 
interval, half shifting to the other. An overall comparison of the groups 
being extinguished under massed and distributed practice appears in 
Figure 105 , which shows that there is a slight tendency for the group 
being extinguished under massed practice to show a more rapid reduction 
in response strength than the group extinguished with distributed trials. 
Figure 10.6 presents a more detailed analysis of the same data vfith the 
performances of different acquisition groups being considered separately. 
In this .figure, the performances of the groups conditioued with distribu- 
ted practice show a wide divergence, with the distributed-extinction 
group displaying very much greater resistance to extinction than the 
massed group. For the group conditioned by massed trials the difference 
is smaller and, it should be noted, in the opposite direction 
The meaning of these data apparently is as follows For subjects ori- 
ginally conditioned with distributed-acquisition trials, the switch to 



c f-5 G-«3 (MS IS -20 21-25 

TRIALS BY FIVES 

FIGURE 10.5. Extinction cunet from Jlcjnohh’ experiment (Figure 30,5) 
plotted separately' for group* conditioned hjr masted and distributed practice. 
The first letter in the label on each cune indicate* condition ol /paring ol 
trial* during ac<lui’*ition (D— distributed; geffitiKil), The second letter 
indicate* condition during extinction (Reynold*, I9t5b). 
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massed practice during extinction produces (1) an increased accumula- 
tion ox inhibition by comparison with the distributed-extinction group, 
and (2) a loss of response strength through generalization decrement. 
For the subjects conditioned and extinguished with massed trials, inhibi- 
tion similarly accumulates at a rapid rate during extinction. But the 
massed-extinction procedure maintains the conditions of distribution 
which existed in training. Under these circumstances response strength 
does not suffer from generalization decrement. The importance of this 
second factor is indicated by the fact that extinction with distributed 
practice occurs slightly more rapidly than with massed practice follow- 
ing training under conditions of massing. 

Various other investigators have made a similar interpretation of the 
role of the spacing of extinction trials and have presented evidence simi- 
lar to that obtained by Reynolds. Teiehner (1952) performed two experi- 
S T ^ ,-f. ’ eons ^ erec l together, provided the data presented in Figure 
10,/ ‘ In tluS Study ’ the bar-pressing responses of rats were first condi- 



tioned at intertrial intervals of a - 

extinguished at intervals of 15 ('fir-a- ° r ^ sec onds. Then they wert 
seconds. As Figure 10.7 shosvs. one S5’^ en l 0I,I ? r) ' 30 ' 45 . 60, or 9( 

' ettect of the distribution oi extinction 
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trials was to produce greater resistance to extinction with increasing 
time between trials This finding illustrates the importance of inhibition 
m the sense in. which we used the term in the previous section Over and 
above this effect, however, there is an increase in resistance to extinction 
which occurs when extinction takes place at the same mtertrial inter* al 
as had existed in training This effect is quite clear m the curves for the 
groups conditioned at the 30 and 45 second intervals Results similarly 
pointing to the significance of a generalization decrement factor ha\e 
been obtained by V P Sheffield (1950) and Stanley (1952) for the 
straight alley and T maze situations, respectively D J Lewis (195G) 
obtained negative results in a study which was well controlled and com 
pletely analyzed statistically The reasons for such discrepancies are not 
clear 


Spontaneous Regression 

With the conception of kinesthetic feedback as an entering explana 
tory wedge, many otherwise puzzling facts about conditioning and ex 
tmction become clear If one considers the stimuli involved in the control 
of the CR, it becomes apparent that, in addition to the CS and the cues 
produced by the act of responding, two other classes of stimulation are 
important (1) the stimulus aftereffects of the UCS and (2) the 
heterogeneous array of stimuli attendant upon being introduced to the 
experimental situation and remaining in it If it is accepted that these 
stimuli in some measure control the occurrence of the CR, any interfer- 
ence with the pattern of these cues will lead to a loss of response strength, 
by the same generalization decrement principle as was employed m the 
analysis of the effects of changing distributions of practice Such a Joss 
could occur without the appearance of a CR, as a sort of “automatic 
decouditiomng” (Razran, 1956) or “spontaneous regression” (Estes, 
1955) 

Among the incidental facts which seem explainable m terms of an 
interference with the cues developed m the course of conditioning arc 
the following (1) When animals are tested for the retention of CR’«, 
there is usually some loss, m spite of the remarkable stability of condi- 
tioning This loss may represent the extent to which the response is 
controlled by the cues in question (2) The beneficial effect of providing 
the rat with a taste of food prior to a maze run rnuj result from the fact 
that this procedure restores an element of the stimulus situation ordunr 
ily lost with the interruption of practice (3) Ivurke (1956) found that 
the resistance of a bar pressing response to extinction was remirka 1>J.\ 
similar when measured in terms of the average nmnbtr of cxpcniin ntal 
sessions required to meet a five minute criterion, although the number of 
responses required ranged from Uss than 10 to more than 150 under van- 
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ous conditions. This suggests that a certain amount of the control of con- 
ditioned behavior resides in the stimuli associated with entering the 
experimental situation, and that this control extinguishes in a fairly 
regular manner as a function of the number of extinction sessions. 

The portion of these stimuli contributed by the mere fact of being in 
the experimental situation presumably would disappear as soon as the 
organism leaves the situation; and the traces of them would be lost 


soon thereafter. Stimulus consequences of the UCS, particularly of a 
noxious one, conceivably might last longer, but they, too, might be ex- 
pected to disappear within a matter of an hour or two at most. These 
considerations suggest that there should be a brief function relating a 
loss in response strength to the time the organism has been away from 
t e experimental situation, and that, after some relatively short period, 
the function should be asymptotic. Meyer (1958) reports a set of results 
w e support this expectation in a limited way. He conditioned rats in 
an avoidance situation and then extinguished them after a wait of 1 
hour i days, 4 days, 8 days, 16 days, or 32 days. The numbers of trials 
o extinction for these six groups, respectively, were: 15.1, 4.5, 7.9, 5.4, 

tt f u ^ a .^ ern *h ese points is what would be expected, 

n ortunately, the differences among them turned out not to be signifi- 

huvvpvp ^ 6Ste - y the metbods °1 analysis of variance. The trend, 
duce the ^ S rea 7 sann " and a study with more subjects might well pro- 
intervals m " m ° re da P^dable way, especially if time 

intervals shorter than one hour were sampled. 

has^rnTfrl^ 61166 “ ^ dissipation of response-produced stimulation 
tied ratt T by ^othkopf (1955) in which he first 

schedule and, later on' sWftedtoTTT ^ ° n & 0Ue ' triaI ' a ' day 

hind this method was ’that with t^? ^ Tbe reasonin " be ‘ 

day traces of thp ™ i ’ ria ^ s oee nrnng at the rate of one a 

p/ rt oi £ Tt d r ie a 

the stimuli produced bv the r P rmance on this second trial because 

would function as external inSb-f 1563 01 \ tbe first trial > if still present, 
ning response. inhibitors and disrupt the conditioned run- 

second response in^e Te^rpart ^tth ^ ^ introduction of tbe 
effects: (1) It will Drovidp i • , tbe es P er nnent will have two 

quences of the response on the firstTrial ° f ^ eonse ' 

second response will be weakened hv ’ ^ Et tbe Same time > ( 2 ) tlie 
erated on the first trial. Since these • i’ eSponse ' produced inhibition gen- 

tion, Eothkopf took measures desired , tleneeS operate in the same direc- 

e signed to separate them. The experiment 
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proceeded as follows training, with trials at the rate of I trial a day, 
continued for 10 days Then, from the eleventh day on, the animals re- 
ceived two trials a day with the trials separated by different amounts 
of time, 5, 15, 30, or 120 seconds Some of tlie animals continued to re- 
ceive two trials a day at the same spacing for the rest of the experiment 
Others were shifted from one interval to another and, m the course of 
the study, had blocks of trials at each of the intervals mentioned above 
The results of the experiment appear in two learning cunes based on 
response latency on the first and second trials separately Figure 10 8 
shows a set of such functions for the groups receiving all of their second 
trials at the same interval following the first The points of importance 
in tins graph are the following (1) As predicted, the second trial laten- 
cies were increased, m a way which depended upon the intertrial interval 
This is most clearly apparent on the first test (2) With practice the 
disruptive effect disappeared By the end of the experiment, the animals 
were responding as if the cues produced by the response on the first trial 
had become a part of the stimulus situation controlling the response on 
the second trial (3) The fact that the 5 second group continues to be 
inferior suggests that, in this group, an additional effect, response pro 
duced inhibition, continued to depress performance Figure 10 9 presents 
results for the groups whose second trials were at several different inter 
vals during the experiment Again it is obvious that practice at a given 
spacing of trials reduced the disruptive effect Successive shifts in the in- 
terval, it will be noted, produced less and less disruption This seems to 
indicate that learning to respond at one interval (and with one set of 
response produced stimuli) generalized to others 


The Generalization-Decrement Hypothesis and Partial 
Reinforcement 

The generalization decrement hjpotbests asumes tint the heightened 
resistance to extinction following partial reinforcement occurs because 
such training increases the similarity in the stimulus situation between 
the learning and extinction, with the result that there is less generaliza- 
tion decrement when extinction begins Specifically, the effects of non- 
rcinforcement persist for partial!} reinforced subjects, in the form of 
a stimulus trace, and become a part of the stimulus situation on subsc 
quent trials Tor continuously reinforced subjects, by contrast, the stimn 
lus traces controlling behavior on each trial after the first alwa>s include 
the traces of the reinforcement Thus, m extinction, when all trials are 
non reinforced, there is less change m the stimulus situation if training 
has been with intermittent reinforcement than if it has been with con- 
tinuous reinforcement The result should be a greater romance to ex! me 
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trials 


FIGURE 10.8. Learning curves showing the effect of introducing a second trial 
at varying times after the first. For explanation, see text (Rothkopf, 1955). 
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mth highly distributed training trials, the superiority of partial rein- 
forcement should disappear In an experiment specifically designed to 
test this hypothesis, Sheffield ran rats down a straight alley to food, 
some were reinforced on every trial, others were reinforced randomly 
on half the trials Each of these major groups was subdivided, half 
received trials at the rate of one every 15 seconds (massed group), the 
other half received trials at the rate of one every 15 minutes (spaced 
group) In extinction, the previously partially reinforced group showed 
superior resistance to extinction only when training had been massed 
There was no difference between the two spaced training groups Similar 
data in support of the generalization decrement hypothesis haie been 
obtained by Rubin (1953) who showed that distributing training trials 
by 40 minutes and also eliminating the complicating effects of secondary 
reinforcement led to greater resistance to extinction following continuous 
reinforcement 

Weinstock (1954) and Wilson, Weiss, and Amsel (1955), on the 
other hand, have obtained evidence against the generalization decrement 
hypothesis The latter investigators attempted to repeat tbe Sheffield 
experiment with different drive reward conditions They found that 
paitial reinforcement during training always led to greater resistance 
to extinction, whatever the intertrial internal Weinstock ran hooded 
rats to food in an L shaped maze under several percentages of reuiforcc 
ment (30, 50, 80, and 100 per cent) and at the rate of one trial a day , 
which would surely be enough to allow for the complete dissipation of 
the stimulus traces of the previous trial Yet in extinction the perform- 
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anee of the subjects, expressed in terms of speed of running, was as 
shown in Figure 10.10. Clearly there were differences in resistance to 
extinction under conditions where the generalization-decrement theory 
says that there should be none. To account for this, Weinstoek proposed 
a variety of interference theory which will be discussed in connection 
with other explanations of the same sort. 


INTERFERENCE THEORY 

The interference theory assumes that extinction occurs when the CS 
comes to elicit a response other than the conditioned response. In its 
original form, the interference theory came from the same sources as 
inhibition theory, from reflex physiology, particularly the studies of 
the reduction of one reflex by the elicitation of another, and from Pav- 
lovian conditioning, in which an interference concept appeared in the 
form of external inhibition. In this connection, Pavlov had pointed out 
that the presentation of an additional stimulus often led to a response 
decrement, that the most effective external inhibitors were strong and 
unusual stimuli which evoked marked investigatory reflexes, and that 
repeated presentation of the extra stimulus led to a progressive loss of 
e 1 itory effect. All of these observations suggest a deeremental 
mec anism m which the CR is weakened by a competing response 
(Guthrie, 193o; Wendt, 1936). 

Other evidence favoring such a view is to be found in a small amount 
! during conditioning and 

Manv Zd V a '“ condltlon ' !d r <*P°use to the conditioned stimulus. 

S:°;Z StUOTh ' SVOke a nat ™> at the outset of condi- 
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Zed eveUd resoe ZZ f ° md ta «periments on condi- 
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tial lid closure. During t°he acquSon oZh T (2 °° mSee - ) P “ r ‘ 
forced by an air-miff acc l m sition of th e conditioned blink, rein- 
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the ringing of a bell. The conditioned^^f 1011 ^ SaUvaiy res P onse t0 
listening movements; the dog turned ° ri f na11 ^ evoked ZT^al 

d its head and pricked up its ears. 
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FIGURE 10 II Antagonistic relation of conditioned response and original bi 
lateral partial lid closure in dogs during conditioning and extinction. Tbe 
bilateral closure (R CB ), present before conditioning, gave way during condi 
Honing to a unilateral closure (CR ) of siDiiIar latency Tbe decrease in tbe 
bilateral response during conditioning, and increase during extinction, reflect 
an interference between the two responses (Hilgard and D G Marquis, 1935) 

The listening movements gradually disappeared as tbe dog came to 
respond to the bell by licking its chops and turning tow ard the food pan 
A more dramatic instance of the same phenomenon was afforded by an 
experiment, also with dogs, m which an electric shock was employed as 
a signal for the presentation of food (Erofeeva, 1916) The struggling 
and yelping elicited by the shock were not compatible with the condi 
tioned responses which were acquired on the basis of food reinforcement, 
and the original responses gradually diminished until after many train- 
ing sessions almost no trace of them remained 
In other cases the interfering response may be one established m 
some earlier learning experience which reappears during extinction Hull 
(1934b) has described an instance of this sort Rats were first trained to 
run down a 20 foot straight alley to recen c food When tho alley w as 
lengthened to 40 feet, they showed a tendency to halt at the 20 foot point, 
hut with continued training this tendency disappeared and the animals 
ran steadily down the entire path During extinction of the later habit bv 
omission of food reinforcement at the end of the alley the tendency to 
halt at tbe 20 foot point ogam reappeared A study by Kellogg and E L 
Walker (1938a) prowdes still another example Dogs were trained to 
lift the right leg to a conditioned stimulus (buzzer) by shocking the 
right foot Then the electrodes for shocking were transferred to the 
left foot, and the conditioning of the left hg response to the same 
buzzer stimulus was undertaken The initial effect of the new situ 
at ion was to intensify the conditioned response of the right leg In tho 
particular apparatus used the dog was able to lift both Kgs together 
by supporting its weight oil a wooden stock enclosing its neck, and this 
double response was common, persisting for os many as 100 trials. Eicntu 
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generalization decrement. Disinhibition was explained in the same way. 
In this case, the new stimulus presented during extinction was assumed 
to lead, through generalization decrement again, to a reduction in the 
magnitude of sIr. This afforded the CR being subjected to extinction the 
opportunity to reappear. Adding the explanatory power of the sIr con- 
cept to that of I a obviously covers many of the most important facts of 
experimental extinction. 


Evaluation of Hull’s Theory 

In spite of these strengths, Hull’s theory has been subjected to very 
harsh criticism. It is demonstrably weak on methodological grounds 
(Koch, 1954) and is also inadequate in the face of certain extinction 
phenomena (Gleitman, Naehmias, and Neisser, 1954). These latter dif- 
ficulties include the following: (1) It is hard to see how this theory can 
explain the extinction of relatively effortless responses, such as the GSR, 
winch presumabiy would develop little if any inhibition. (2) It is equally 
lmcult to understand the complete resistance to extinction of certain 
very effortful responses, such as jumping a hurdle to escape shock. (3) 
xtmctmn at the rate of one trial a day is probably incapable of treat- 
tl n m ie ory. (4) The facts of latent extinction (see below) and 

arenaH^ 1 ° extm ^ on without responding (for example, under curare) 
are particularly embarrassing. (5) During extinction, the subject is very 

do^slrth y r t ^ 0lding ‘ he conditiMeii response, but is actively 
(ood°show f 75 tra “ ea t0 open a door 011 signal ‘° obta “ 
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("Wendt 1 <y>e i tk\ , n , p ° tbe door * an d racing about the cage 
™ ^ to bring in” Le° f Jth“eo“ OWi ° g 
theory & eTCn ^ith th^^fT ^ t0 reco S nition that Huff’s inhibition 

bitioZ'is e r t raat;^L d natnf rf ““ ”°T « 
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Sources of Motivation and Reward in Extinction 

himself to the two^most^diffil °u Conditl0necI inhibition, Huff addressed 
of extinction, the ori<dn of the h!t° faeiag an interference theory 
by which such a response eontl 1 , er ^ eriQ = response and the mechanism 
Huff came, that the response was to rengtlleiled - Tlie answers to which 
and that reinforcement rionn i a dnve s t ate generated in extinction 
rcemeut depends upon the reduction of this drive, provide 
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instructive leads, in spite of the fact that the particular drive and rein 
foreement mechanism Holl proposed do not seem generally applicable 
The suggestion is that it may be fruitful to look for other possible motives 
and rewards for interfering behavior in the extinction situation Within 
this general orientation some of the possibilities which suggest themsehes 
are the following (1) Responses produced by frustration, reinforced by 
the escape from frustration , (2) the I R I R reduction hypothesis proposed 
by Hull, (3) investigatory behavior, stimulated by novel stimuli, (4) 
interfering responses resulting from persisting stimuli such as those ans 
ing from an irritating injury, distension of the bladder, and sexual ex- 
citement (Pavlov, 1927, p 47), and (5) in fear conditioning experiments, 
responses based on fear, reinforced by fear reduction Wc shall deal with 
this possibility first 


The Extinction of Avoidance Behavior 
As we have seen, avoidant behavior is often highly resistant to extinc 
tion Yet there are individual exceptions to this, and with an embarrassing 
frequency, such behavior is found to extinguish after just one or two ex 
tinction trials Occasionally extinction even occurs without any non rein 
forced trials at all (Moyer, 1958) This fact, in and of itself, raises the 
crucial question Avoidance behavior is generally considered to be moti 
vated by an acquired drive of fear Does this no trial “extinction" mean 
that the fear has somehow disappeared with the end of training f It seems 
unlikely that this is the case because animals which “extinguish" pre 
cipitously in this way may just previously have been thoroughly condi 
turned, suggesting the existence of a high level of fear 

The alternative to the unlikely explanation in terms of the spontaneous 
reduction of fear is that the fear is still there, hut that the organism has 
learned some other response to reduce it In other words, extinction occurs 
because the organism has acquired interfering responses The results 
obtained by Page (1955) may be cited in support of bucli an interprita 
tion In an earlier study. Page and Hall (1953) hod trained rats to go 
from one side of an apparatus to the other to avoid shook and following 
training to a criterion, extinguished the rats in two different wnjs Ono 
(control) group received ordinary extinction trials beginning imme- 
diately after conditioning For the other (experimental) group the first 
5 trials were blocked These animals were put m the starting box on each 
trial anil restrained there for 35 seconds Following these 3 trials the 
experimental animals were extinguished in the same way as the control 
group The number of trials required to produce extinction was 33 for the 
control group and 13 for the experimental group Quite clrarlj the re- 
straining procedure had prodnetd some measure of extinction The qtics 
tion is whether this extinction consisted m the reduction of ftar or 
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instructive leads, in spite of the fact that the particular drive and rein- 
forcement mechanism Hull proposed do not seem generally applicable. 
The suggestion is that it may be fruitful to look for other possible motives 
and rewards for interfering behavior in the extinction situation. Within 
this general orientation some of the possibilities which suggest themselves 
are the following: (1) Responses produced by frustration, reinforced by 
the escape from frustration; (2) the Ir, iR-reduction hypothesis proposed 
by Hull; (3) investigatory behavior, stimulated by novel stimuli; (4) 
interfering responses resulting from persisting stimuli such as those aris- 
ing from an irritating injury, distension of the bladder, and sexual ex- 
citement (Pavlov, 1927, p. 47), and (5) in fear-conditioning experiments, 
responses based on fear, reinforced by fear reduction. We shall deal with 
this possibility first. 


The Extinction of Avoidance Behavior 

As we have seen, avoidant behavior is often highly resistant to extinc- 
tion. Yet there are individual exceptions to this, and with an embarrassing 
frequency, such behavior is found to extinguish after just one or two ex- 
tinction trials. Occasionally extinction even occurs 'without any non-rein- 
forced trials at all (Moyer, 1958). This fact, in and of itself, raises the 
crucial question : Avoidance behavior is generally considered to be moti- 
vated by an acquired drive of fear. Does this no-trial “extinction” mean 
that the fear has somehow disappeared with the end of training t It seems 
unlikely that this is the case, because animals which “extinguish” pre- 
cipitously in this way may just previously have been thoroughly condi- 
tioned, suggesting the existence of a high level of fear. 

The alternative to the unlikely explanation in terms of the spontaneous 
reduction of fear is that the fear is still there, but that the organism has 
learned some other response to reduce it. In other words, extinction occurs 
because the organism lias acquired interfering responses. The results 
obtained by Page (1955) may be cited in support of such an interpreta- 
tion. In an earlier study, Page and IlaJI (1953) had trained rats to go 
from one side of an apparatus to tlie other to avoid shock and, following 
training to a criterion, extinguished the rats in two different ways. One 
(control) group received ordinary extinction trials beginning imme- 
diately after conditioning. For the other (experimental) group the first 
5 trials were blocked. Those animals were put in the starting box on each 
trial and restrained there for 15 seconds. Following these 5 trials, the 
experimental animnls were extinguished in the same way ns the control 
group. The number of trials required to produce extinction "as 38 for the 
control group and 13 for the experimental group. Quite clearly the re- 
straining procedure had produced some measure of extinction. The qtirs- 
tion is whether this extinction consisted in the reduction of fear, or 
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■whether it involved learning a new interfering response. Page’s later 
(1955) experiment shows that at least part of what is involved is the 
second of these processes. This stndy was a repetition of the earlier 
one in its first stage, and the resnlts were the same. The numbers 
of trials to extinction for the control and experimental groups were 
30 and 8. respectively. In a second stage of the experiment, the rats of 
both groups were trained to run from the previous goal compartment to 
the starting box to obtain food, the reasoning being that performance 
under these circumstances would reflect residual fear still conditioned to 
the starting box stimuli. If the animals were still afraid of the starting 
box, they would hesitate before going there to get food. An adequate test 
of this idea required the use of a second control group which had not 
previously been shocked in the experimental situation. Results of the 
experiment suggested that, so far as fear was concerned, the blocking 
procedure had, if anything, increased it. Average latencies (geometric 
means) for the first 5 training trials estimated from a graph published 
by Hall were approximately 25, 60, and 110 seconds for the control, 
normal extinction, and blocked groups respectively. This increase in fear 
with blocking seems to agree with the report of Solomon and Wynne 
(1954) that animals showing a long latency (which would be like re- 
straining them) are apt to respond very rapidly on the next several 
trials, as if their fear had increased. 

The important fact for our present discussion is the evidence in this 
study that fear had not extinguished under the experimental condition, 
although the avoidant response had. This means that the fear, which 
originally motivated avoidant behavior, was at least available to instigate 
other habits, and that fear reduction might conceivably have reinforced 
them. In short, the possibility exists that, following avoidance condition- 
ing, interfering responses may be reinforced in exactly the same way as 
the response subjected to extinction originally was. 


At least two objections may be raised to this explanation: (1) It is 
entirely ad hoc . 1 et it probably makes sense to anyone wlio has run rats 


in an avoidance-training experiment. It is fairly common, in extinction, 
to observe an animal, apparently paralyzed with fear, frozen in a position 
which makes the avoidant response impossible. And if the animal is still 
in the same position at the start of the next trial, it will probably not 
respond. It is as if the animal had been reinforced for immobility by 
a\oiding the shock. Animals responding this way in extinction tend to 
extinguish precipitously and sometimes after a very few trials. (2) There 
is no precise explanation of the source of the interfering responses In 
this connection it is probably important to recognize that the typical 
avoidance-training situation is poorly suited to the discovery of the origin 
of such responses Denny, Koons. and Mason (1959), however, have "ob- 
tained data v Inch suggest that, sometimes, these interfering responses 
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may occur as a result of stimulus generalization. These investigators 
trained rats to jump out of a box to avoid shock. The escape area, for 
different groups, was either a box like the starting box where shock was 
administered or an open area which was perceptually much different from 
the starting box. Extinction was found to be faster when the start and 
escape areas were similar. It, thus, may be that "safe" responses occur- 
ring in the similar escape box generalized to the starting situation and 
interfered with the avoidance response. 

There is also one more general point which needs to be made. The use- 
fulness of this specific explanation is probably limited to the extinction of 
responses based on negative reinforcers. On common sense grounds it 
seems unlikely that interfering responses in a situation employing a food 
reward can ever be reinforced in extinction in the same way as the cor- 
rect response was during training. This suggests that, if the interference 
hypothesis outlined above is to have general applicability, it will be neces- 
sary to discover, in the extinction procedure, still other sources of motiva- 
tion and reward. 


The Frustration Hypothesis 


At the present time frustration produced by the omission of reward 
seems to provide the most promising lead as to the source of interfering 
responses in the extinction of positively reinforced habits. The frustra- 
tion hypothesis differs from the explanation in terms of anxiety described 
in the previous section in that the motivating and rewarding processes are 
peculiar to extinction ; they are not carried over as a residual from train- 
ing. In the application of the frustration hypothesis to extinction, the 
assumption has been that frustration provides the motivation for inter- 
fering responses, and that these responses are strengthened by escape 
from frustration. The assumption that extinction sometimes creates a 
motivational state of frustration has led to predictions of three general 


kinds: (1) Since it is assumed to be a motive, frustration should be ca- 
pable of energizing behavior; that is, performance following frustration 
should be more vigorous than would have occurred without it. (2) Most 
if not all motives have associated drive stimuli. Frustrating an animal, 
therefore, might be expected to produce cues which could be made the 
condition for some learned act. (3) Since frustration, Introspective^ is 
aversive, analogues to escape and avoidance should be demonstrable. 

Possibly the first clear demonstration of the energizing function of 
frustration on learned behavior was that of Anisel and Roussel (ISM). 
These investigators employed a type of apparatus winch might ho called 
a two-stage straight runway. The find slope led to the first poa! box; he 
second sta-e led to the second goal box. Hungry rats first ran from _tbe 
startin', box to the first goal bos, in which they were rewarded with food 
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(or not regarded when the object was to frustrate them). Then they ran 
the second stage of the maze to a second goal box where they were always 
rewarded. The standard procedure consisted of first running a long series 
of trials with reward available in both goal boxes, and then switching to a 
situation in which reward was omitted in the first goal box on half the 
trials (that is, to partial reinforcement) . The assumption that non-reward 
is frustrating leads to the prediction of faster running in the second stage 
of the maze following non-reinforced trials. The results of the experiment 
vrhich appear in Figure 10.12 supported this expectation. Alternate e 


(319) 



FIGURE 10.12. Graph showing the effect of frustration upon the running time 
of rats (Amsel and Roussel, 1952). 


explanations which might he developed, capitalizing upon a lovering of 
motivation produced by the consumption of the food pellet in the first 
goal box (Seward, Pereboom, Butler, and Jones, 1957), seem to have 
been ruled out by later experiments (Wagner, 1959). 

The second implication of the assumption of a frustration drive that 
frustration will have associated drive stimuli which can serve as cues for 
learning, has also been investigated. Amsel and J. Ward (1954) per 
formed a series of experiments in which the procedure initially was like 
that just described In a final phase of the experiment, however, they 

t t v eCOn u Sta ? f the d0Uble runwa >- ^placed it ; it h a 
T In thls phase of the experiment, the animal first ran from the 
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starting box of the maze to the first goal box where it found food or no 
food, depending upon the trial. Then it was allowed access to the T-maze. 
Following reinforcement a left turn led to food; but following non-rein- 
forcement (frustration) a right turn led to food. The animals learned 
this discrimination in five somewhat different experiments. Whether such 
data provide evidence favoring the hypothesis that frustration stimuli 
can control behavior remains a question. Tyler, Marx, and Collier (1959) 
have argued that the stimulation supplied by the presence or absence of 
food at the end of the first maze segment, and by the proprioceptive con- 
sequences of eating, could easily guide the animals’ subsequent right or 
left turn, and that no added frustration concept is necessary. To test this 
hypothesis these investigators performed an experiment patterned after 
that of Amsel and Ward. Among the groups of subjects in the experiment 
were one which found no food on the “frustration” trials and another 
which found inaccessible food on the same trials. Tyler, Marx, and Collier 
proposed that this latter group, if anything, should have been more frus- 
trated than the no-food group, that the frustration stimuli should have 
been more intense, and that the learning of this group should have been 
superior to that of the no-food group. As it turned out, however, the 
no-food group was superior. This led to the conclusion that the discrimina- 
tion between available and unavailable food was more difficult than that 
between food and no food, and that performance suffered under the more 
difficult discrimination. Such an explanation does not invoke the concept 
of frustration stimuli at all. 


Frustration in Extinction 

Returning to the question of whether frustration contributes to re- 
sistance to extinction, the experiment of Adelman and Maatseh (3956) 
provides a demonstration that it does. In this study, the response in- 
vestigated was that of jumping out of a box 10 inches high for rewards 
of three different kinds., satisfaction of curiosity, food, and release from 
frustration. For the food-reinforced group the procedure was as follows : 
After 37 habituation trials without food and without an opportunity to 
escape from the box, these animals received 30 trials in which jumping 
out of the box, and up onto a ledge, was reinforced with food. The animals 
in the group rewarded with the satisfaction of curiosity were treated in 
the same way, except that there was no food; that is, there were 37 
habituation trials and 30 trials in which the rats jumped out ot the box. 
These latter trials, however, were unreinforeed eieept bp escape from the 
bos and the possible satisfaction of exploratory and curiosity drives. 'Hie 
frustrated subjects were treated somewhat differently. Instead of 3i 
habituation trials they received 37 trials in a runway which terminated 
in the jump boa. On these trials they were reinforced with food, id order 
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to provide a basis for frustration later on in the experiment. They 
-n-ere not allowed to jump out of the box. In the later phase, the frustra- 
tion animals, as usual, ran the straight alley into the box; but m this 
case they found no food there. They were confined for 5 minutes and 
then allowed to escape from this theoretically frustrating situation by 
jumping out of the box. 

There were vast differences in the effectiveness of the three reinforcers 
used in this experiment. One reflection of this is in terms of a a erage laten- 
cies during acquisition. For the curiosity-rewarded, food-rewarded, and 
frustration groups they were, respectively, 168, 20, and 5 seconds. There 
were even greater differences in extinction behavior. The curiosity-re- 
warded animals were, in effect, “extinguished” at the end of the acquisi- 
tion phase of the experiment ; the food-reinforced animals required about 
60 non-reinforced trials to reach a criterion of extinction. The animals 
rewarded by escape from frustration, however, showed no signs of extinc- 
tion in 100 trials. This attests to what appears to be the tremendous 
strength of escape from frustration as a reinforcement, a point to which 
we shall return later. 

One matter upon which Adelman and Maatsch did not comment was 
the runway behavior of their animals. Since frustration is aversive, it 
might be supposed that there would be some tendency to avoid the frus- 
trating situation. "W. B. Holder, Marx, E. E. Holder, and Collier (1957) 
report such an effect. These investigators used a maze situation not unlike 
that employed by Amsel and Roussel (described above) and a procedure 
similar to Amsel and Roussel’s frustration procedure. They ran three 
different groups of rats down a 90-inch runway to food, delaying the 
different groups for different amounts of time halfway down the runway. 
One group was delayed for 1 second (essentially no delay) ; two other 
groups were delayed for 15 and 45 seconds. The effect of the delay was 
to retard both starting time and running time in the maze section before 
the delay, as if the delay were an anticipated noxious aversive s tim ulus. 
In the second maze segment, the effect was the opposite ; both starting 
time and running time were faster. The second result was similar to that 
of Amsel and Roussel, with the difference that, in this study, the am ount 
of frustration apparently increased with practice. This last result sug- 
gests that either the frustration drive or the responses elicited by it can 
be conditioned to stimuli in the learning situation. 

Some of the most impressive evidence that the frustration produces 
very- persistent learning comes from the work of Maier on “abnoral fixa- 
tions” m rats. In these studies the procedure is to confront the rat in the 
Lashley jumping stand (Figure 10.13) with an insoluble problem Two 
stimuli (for example, a white circle on a black background and a black 
circle on a white background) are placed randomly right and left in the 
apparatus, and responses to each are randomly reinforced half the time. 
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Under these circumstances, any course o£ action adopted hy the rat leads 
reta or Lent on halt the trials. On the other half of the _ trials, the 
to reintorce door an a falls into a net. Maier regards 

ammal jump » £rustr ating. Some evidence that he is right ap- 

tn the fact that the animals soon develop a marked tendency not to 
pears » the fa “““ appens , they are forced to jump by electric shock 

Sd to ihTfL o h : by an at jet applied to the base of the tad. These 



FtGURE 10.13. A 

in .be openies .1 - 

(Ehrenfreund, 1949). 

unpleasant stimuli probably make ^develop 

t any event, the rats *"*”?*^ he situation. Most often these 
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right or left unless special “therapeutic” techniques are introduced 
(Maier and Ellen, 1952; Maier and Klee, 1945). This is in spite of the 
fact that other evidence may indicate that the animals actually have 
solved the discrimination problem. This persistence of the maladaptive 
behavior in spite of the fact that the animal apparently “knows” the 
solution to the problem is why the fixated behavior is called “compul- 
sive.” Fel dm an (1953), for example, compared the latencies of fixated 
jumps to the positive and negative cards and found the latencies very 
much longer to the negative cards after a certain amount of training 
(Figure 10.14). It has also been found that the animals can respond 
correctly once the problem is made soluble if the situation is changed so 
that a response to the correct stimulus does not differ too much from the 
compulsive, fixated response (Ellen, 1956). 



FIGURE 10.14. Evidence for the compulsive nature of fixations. Although the 
rat jumps in a rigidly stereotyped (“compulsive”) way, it is obviously dis- 
criminating correctly between positive and negative stimuli as is evidenced 
by the latencies of response to these two stimuli. Stage 1 was the frustration 
stage of the experiment designed to produce fixated behavior. In Stage 2, the 
problem was solvable, hut all subjects which solved it were discarded The 
data, thus, are for animals behaving in a compulsive way. Subjects not re- 
sponding in 30 seconds were shocked with an electric “wand” to make them 
jump (Feldman, 1953). 


Naier regards abnormal fixations as a natural (that is, unlearned) con- 
sequence of frustration. It seems possible, however, that the great re 
sistanee to change of the fixated response comes about because" Maier ’s 
experimental situations are arranged so that three separate factors are 
all made to operate in favor of resistance to extinction: (1) Original 
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learning involves partial reinforcement (Wilcoxon, 1952) ; (2) a noxious, 
anxiety-evoking stimulus is one of the sources of motivation, an arrange* 
ment which often leads to very persistent behavior; and (3) frustration 
from non-reinforcement and frustration reduction, the factor which 
Maier stresses, may contribute to the persistence of the response. It seems 
quite possible that a series of jumps (say) to one side of the apparatus, 
occurring at first more or less by accident, is powerfully reinforced by 
these three sets of factors. After that, intermittent reinforcement of the 
same powerful sort would continue to support the fixated response. This 
argument, if correct, reduces the abnormal fixations to the expression of 
perfectly normal processes. There is, however, no suggestion in this that 
the behavior of Maier's rats is not analogous to human compulsive be- 
havior. It is entirely possible that it is, because human compulsions, too, 
are no doubt established by normal learning principles. 


The Frustration Hypothesis and the Effect of Partial 
Reinforcement 


The explanation of the partial-reinforcement effect in terms of frustra- 
tion relies upon the fact previously demonstrated, that frustration is ca- 
pable of strengthening the response which follows non-reinforcement. 
Since partial reinforcement introduces frustration on non-reinforeed 
trials, this should strengthen behavior during training ; and, if the frus- 
tration drive is conditionable, as there is reason to believe it is, this con- 
ditioned frustration would carry over into extinction and increase the 
persistence of behavior there. The first part of this interpretation raises 
a problem: the usual outcome of partial-reinforcement studies is that 
behavior during training is slightly weakened, whereas frustration theory 
seems to require a strengthening of the response as a result of heightened 
drive. How is the low performance level during training to be accounted 
for? Two lines of reasoning, one hypothetical, the other factual, may be 
offered by the frustration theorist in response to this question: (1) In- 
creased drive theoretically would increase the strength of the organism’s 
responses indiscriminately. If one effect of partial reinforcement is to 


evoke competing responses, these would be strengthened and the response 
being measured thereby weakened. (2) In some cases the predicted 
strengthening effect has occurred. Goodrich (1959) and Weinstock (19 j 8) 
have both obtained superior performance with partial reinforcement. 
■Weinstock ran 6 groups of approximately 40 rats each for a total of 203 
training and 60 extinction trials in a runway at the rate of one trial a 
day. For different groups, acquisition was under six different schedules 
of reinforcement. The results of this investigation, thus, provide some of 
our most substantial data on the effects of partial reinforcement in the 
runway situation. In terms of numbers of non-reinforced trials in brocks 
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of 12 trials, the schedules of partial reinforcement were 0 (continuous 
reinforcement), 2, 4, 6, 8, and 10. The response measure employed was 
speed of leaving the starting box. Briefly the results were these. (1) 
Early in learning, performance was superior for the groups receiving rel- 
atively consistent reinforcement. This result conforms to that obtained in 
the great majority of other experiments, which have typically used rela- 
tively brief periods of training. (2) Later in training, however, the groups 
receiving infrequent reinforcement performed best. (3) Extinction be- 
havior conformed to the usual pattern. Groups with low proportions of 
reinforced trials during training continued, throughout extinction, to 
run faster than groups with high frequencies of reinforcement during 
training. 

The essentials of the explanation of these results, proposed both by 
Goodrich and by Weinstock, are as follows: The effect of frustration is to 
increase motivation. Early in training, before the running response is 
firmly established, the effect of this motivational increase is mainly to 
strengthen irrelevant and interfering responses produced by frustration. 
This accounts for the lowered performance under intermittent schedules 
early in training. “With increased practice, however, the interfering re- 
sponses tend to disappear because they are never reinforced, and the 
animal’s behavior is limited fairly strictly to the running response which 
is the one most consistently reinforced. When this happens, the motiva- 
tion produced by frustration has only the correct response to act on, with 
the result that partial reinforcement now improves performance. In ex- 
tinction, animals previously trained with continuous reinforcement are 
frustrated for the first time, producing interfering responses long since 
abandoned by partially reinforced subjects. As a result, the running re- 
sponse of the former group extinguishes quickly. 

The acceptance of such an explanation entails two difficulties. One is a 
general methodological problem encountered by any version of an inter- 
ference theory. The interfering responses proposed in an explanation of 
this sort are in the nature of intervening variables, seldom being observed 
directly or manipulated experimentally. The study of Adelman and 
Maatsch (1955), however, suggests that this difficulty is not insurmount- 
able. These investigators ran 30 rats in a straight alley to a food reward 
and then extinguished them by three different procedures. One subgroup 
of 10 rats received a normal extinction series in which they were simply 
confined in the goal box for 20 seconds following the response. A second 
gioup was allowed to jump out of the goal box, theoretically escaping 
from a frustrating situation. The animals in the third group were allowed 
to escape from the goal box by running back into the alley through the 
door to the goal box which had been left open. This group was cabled a 
recod group. Results of the experiment in terms of runnum time ap- 
pear m Figure 10.15. The response elicited in the goal box" obviously 
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makes a great deal of difference in determining resistance to extinction. 
In the recoil group the response following non-reinforcement is directly 
incompatible with running, and extinction proceeds very rapidly. For 
the jump group, making a compatible response, extinction is very greatly 
retarded. The normal group, whose responses were uncontrolled, fell in 
between. 



FIGURE 10.15. Behavior in extinction ns determined by the response which 
occurred to the empty goal box (Adelman and Maatsch, 1955). 

These results lend support to an interference interpretation of extinc- 
tion, but they also raise the other question: How are responses (even in- 
compatible responses) elicited in a goal box able to interfere with the 
learned response, which is measured elsewhere in the learning situation ? 
Obviously some mechanism must be supplied to make the interferences 
occur at points in learning when they actually interfere. It is to be noted 
that this problem is exactly the same as the one previously encountered 
in connection with incentive motivation. Adelman and Maatsch report, 
in the case of their jump group, that they sometimes saw jump responses 
occurring in the starting box. This is some evidence that responses to non- 
reinforcement do generalize to other portions of the maze. It is of some 
interest that this took place in spite of the fact that the goal hox and 
starting box were quite different, suggesting that general situational 
cues, or perhaps internal cues, provided the basis for generalization. 


EXPECTANCY THEORY 

Tolman’s original statement of the expectancy hypothesis was that con- 
ditioning consisted of the establishment of an expectancy that the UCS 
will follow the CS, and that for extinction to occur, all that was necessary 
was to prove to the animal that this correlation no longer existed. As with 
other aspects of expectancy theory, the development of this counter- 
expectancy was assumed to be a perceptual matter and not to require the 
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performance of an overt response. Prom this general orientation, two 
hypotheses have emerged which have guided much of the research con- 
ceived in the expectancy framework: (1) the discrimination hypothesis, 
and (2) the prediction that a latent-extinction effect comparable to latent 
learning is possible. 


Tlie Discrimination Hypothesis 

In the same paper in which they offered the response-unit hypothesis to 
account for the effects of partial reinforcement upon extinction, Mowrer 
and Jones (1945) also outlined what has come to be known as the dis- 
crimination hypothesis. According to this hypothesis, extinction is re- 
tarded by any acquisition procedure which makes it difficult for the 
subject to distinguish between training and extinction. The conditions of 
training which have since come to be regarded as making such discrimina- 
tions easy are (1) continuous reinforcement, (2) regular patterns of re- 
inforcement in the partial-reinforcement situation, and (3) alterations in 
the stimulus conditions at the beginning of extinction. 

1. Tlie Discrimination Hypothesis and Partial Reinforcement. One of 
the original arguments in favor of the expectancy theory was based on the 
effect of partial reinforcement. In terms of the discrimination hypothesis, 
this argument is that extinction following partial reinforcement is slow 
because the expectancy developed during learning includes the subex- 
pectancy that reinforcement will not occur on every trial. For this reason 
the beginning of extinction, after which reinforcement never occurs, is 
more difficult to perceive than when extinction occurs after consistent 
reinforcement. Although this reasoning leads to predictions which fit the 
facts, it provided no critical support for expectancy theory because other 
theories were also able to explain the effects of partial reinforcement with 
the aid of the concepts of secondary reinforcement and generalization 
decrement. Probably for this reason, the expectancy theorist was led to 
look elsewhere for evidence in support of his position. 

2. The Effect of Patterned Reinforcement. Within a partial-reinforce- 
ment schedule, reinforcements may be administered either irregularly or 
regularly, according to some pattern, such as on every second, third, or 
fourth trial. In general, extinction following irregular reinforcement is 
much slower than following regular reinforcement. Longnecker, Kraus- 
kopf. and Bitterman (1952) conditioned the galvanic skin responses of 
two groups of human subjects, using two different partial-reinforcement 
schedules. One schedule omitted shock on half of the trials in an irregular 
order ; the other schedule consisted of a regular alternation of reinforced 
and non-reinforeed trials. Following 22 trials administered according to 
these schedules there were 30 extinction trials. Three important results 
came from this study: (a) During training, the performance of the ran- 
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dom group was somewhat superior to that of the alternation group, al- 
though this effect was less than has been obtained in other studies, ’(b) 
During training, the magnitude of the conditioned GSR alternated ap- 
propriately in the case of the regular alternation group, being large on 
shock trials and small on non-shock trials. No such effect appeared in the 
random group, (c) During extinction, performance was much superior in 
the group conditioned on the random schedule. Tyler, Wortz, and Bitter- 
man (1953) and Capaldi (1958) have obtained essentially the same re- 
sults in experiments where rats were required to run down a straight 
runway and then jump to a goal box. 

The conditioned eyelid response also displays greater resistance to ex- 
tinction following an irregular schedule of reinforced trials than follow- 
ing a regular schedule, at least when the measure of performance is the 
percentage of conditioned responses (Kimble, Mann, and Dufort, 1955). 
With measures based upon magnitude of the response the extinction 
curves have been found to differ in form following irregular and regular 
(single alternation and double alternation) reinforcement, making re- 
sistance to extinction difficult to compare (Grant, Riopelle, and Hake, 
1950). In the eyelid conditioning experiment, there has been no tendency 
for the conditioned response to follow the pattern of reinforced and non- 
reinforced trials in spite of the fact the subjects typically are able to 
describe the sequence. The reason for this may be that the short intervals 
usually employed in eyelid conditioning studies effectively preclude the 
occurrence of mediating responses necessary for such matching of re- 
sponse to reinforcement condition. 

3. Changes in the Learning Situation. A final line of evidence favoring 
the discrimination hypothesis comes from experiments directed against 
the explanation of the effect of partial reinforcement in terms of sec- 
ondary reinforcement (Bitterman, Fedderson, and Tyler, 195 3; Elam, 
Tyler, and Bitterman, 1954). In the first of these experiments rats were 
trained, on a combination elevated runway and single window jump 
stand, to run to the end of the end of the runway and jump to food. They 
were reinforced on 50 per cent of the trials and not on the others on an 
irregular pattern. During the acquisition phase of the experiment, there 
were two main groups. One, a discrimination group, received reinforce- 
ments and non-reinforcements, respectively, in goal boxes of different 
colors (black or white). The other main group, a nondiscrimination group, 
entered the same goal box on both reinforced and non-rein forced trials. 

In extinction each of the main groups was divided into two subgroups. 

One of these, a secondary -reinforcement subgroup, was extinguished us- 
ing the goal box previously associated with reinforcement. The other 
group, a non-secondary-reinforcement subgroup, was extinguished with 
the previously negative goal box in the case of the discrimination group, 
and with a new goal box, opposite in color to that used in training, in the 



320 CONDITIONING AND LEARNING 

case of the nondiscrimination group. The results obtained from the four 
groups thus produced appear in Table 21. Considering first the results 

TABLE 21 

Geometric Mean of Time per Trial (seconds) in 
15-day Extinction Period 

(Bitterman, Fedderson, and Tyler, 1953) 




Extinction 

Condition 



Secondary 

Non-secondary 



Reinforcement 

Reinforcement 

Training 

Discrimination 

38 

30 

Condition 

Nondiscrimination 

27 

40 


for the nondiscrimination groups, it is clear that the usual effect of sec- 
ondary reinforcement appeared. The rats run to the previously reinforced 
goal box ran faster. In the case of the discrimination groups, however, 
the results were the opposite. Resistance to extinction was reduced by 
using the previously reinforced goal box. 

The discrimination theory handles these results, without reference to 
the concept of secondary reinforcement, as follows : For the nondiscrimi- 
nation groups, the introduction of the new goal box changes the situation 
markedly, leading to easy discrimination of the difference between train- 
ing and extinction. For the discrimination groups, entering the previ- 
ously reinforced goal box, where there is now no food, is a much more 
“impressive” thing to the rat than entering the always negative goal box 
where there never was any food. This again makes it easy for the rat to 
recognize that some important change in procedures has occurred. 


Latent Extinction 

Latent extinction (sometimes called indirect extinction, goal extinction 
or non-response extinction) refers to a reduction in response strength 
achieved without the occurrence of the response itself, by exposing'the 
animal to the goal situation in the absence of reinforcement. It is the 
parallel, m extinction, of latent learning discussed in Chapter 8 Latent 
extinction seems unexplainable in terms of inhibition theory but offers 
no difficulty to expectancy theory. Experimental demonstrations of latent 
extinction typically begin by teaching the animal some response such as 
limning a straight maze (Seward and Levy, 1949), pressing the bar in the 
Skinner box (Coate, 19o6), making the correct choice in a brightness dis 
crimination situation (Deese, 1951), or runnhm a maze fW T, p d i 

IMas, IKTMMTb, MS7., 1M* 1955). anTaMn 

an -PanmenM group arc p ,aoed dlrectly int0 fte efflpty ^ ” 
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ber of times. Those in a control group are placed elsewhere, without re- 
ward, or merely handled. The demonstration of latent extinction consists 
in the fact that the subjects in the experimental group reach a criterion of 
extinction more rapidly than animals without the latent-extinction experi- 
ence. Molts’ experiment is typieal. Sixty rats received 40 trials in a 
T-maze and then were extinguished in one of three ways. One (control) 
group was subjected to ordinary extinction. The other two groups re- 
ceived four one-minute latent-extinction trials in the goal box. Por one 
of these groups the food cup, a powerful secondary reinforcer, was pres- 
ent during latent extinction. Por the other latent-extinction group, it was 
not. In subsequent tests these groups displayed quite different resistance 
to extinction. The numbers of correct responses required to meet a cri- 
terion of extinction were 4.7 for the control group, 5.4 for the group given 
latent extinction without the food cup, and 2.2 for the group given latent 
extinction with the food cup. The only significant differences were be- 
tween the group subjected to latent extinction with the food cup present 
and the other two groups. 

The results of this experiment are important in the following ways: 
(1) They provide a typical illustration of latent extinction. (2) They 
suggest that one of the important variables determining whether or not 
latent extinction occurs is the discrirainahility of the positive and negative 
goal boxes. The evidence for this is in the fact that the performances of 
the control group and the no-food-cup group were not different. Presum- 
ably this is because the positive goal box without the food cup is indis- 
tinguishable from the negative goal box which never contained one. 
Studies which failed to obtain latent extinction (Bugelski, Coyer, and 
Rogers, 1952; Sebarloclc, 1954) have employed situations where this dis- 
crimination was difficult to make. Denny and Ratner (1959) found that 
the elimination of distinctive environmental cues destroyed the phe- 
nomenon of latent extinction. (3) They indicate the possibility of explain- 
ing latent extinction in terms of the loss, through extinction, of the value 
of goal-box stimuli as secondary reinforcers. 

The results of an experiment by Moltz and Maddi (195G) seem to attest 
to the essential validity of such an interpretation. These investigators gave 
60 rats training in a straight alley runway with a food reward. In this 
part of the experiment the animals received 7 reinforced and 3 non-rein- 
forced trials per day. The goal boxes on the two kinds of trials wen? dis- 
tinctly different. The purpose of this training was to provide the goal-box 
Stimuli with strong secondary reinforcing power. In the second part of the 
experiment, half of the animals received four latent-extinction trials in 
the previously positive goal box. Different subgroups received these trials 
under 0 22, or 44 hours of food deprivation. The other half of the animals 
were divided into subgroups and subjected to the same deprivation condi- 
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tions, but they were given no latent-extinction training. Finally all 
animals received 15 trials in a T-maze with the positive and negative goal 
boxes placed on the two ends. The results obtained in this portion of the 
experiment appear in Table 22. It will be noted that the control groups 

TABLE 22 

Mean Numbers of Responses to Previously Positive Goal Box 


for Each Group 
( after Moltz and Maddi, 1956) 


Deprivation During 
Experimental Treatment 

Experimental Groups 
( Latent Extinction) 

Control Groups 
(No Latent Extinction) 

0 hours 

9.30 

9.30 

22 hours 

7.10 

9.90 

44 hours 

6.30 

9.80 


all showed evidence that the positive goal box was secondarily reinforcing. 
The latent-extinction procedure, however, had destroyed the secondary 
reinforcing value of the goal box, except for the satiated group. 

Studies of latent extinction in the Skinner situation have also con- 
sisted of direct attempts to extinguish the secondary reinforcing value of 
the stimuli associated with the delivery of primary reinforcement. Ratner 
(1956) 3 trained rats to press a bar to obtain water which was delivered 
with the characteristic click of the magazine. Following this training, 
different groups of animals received three different treatments. One ex- 
perimental group (click group) was subjected to a latent-extinction pro- 
cedure in which the empty magazine was operated repeatedly to extin- 
guish responses to the click. The bar was absent from the box during this 
procedure. A second group (no-elick group) was placed in the Skinner 
box for the same amount of time as the click group, with the bar removed. 
In this group, however, the magazine did not operate, and the animals’ 
approach responses were extinguished in the absence of the click. A third 
(control) group was merely handled during the time the two experi- 
mental procedures were carried out. On the following day all animals 


TABLE 23 


Mean Number of Bar Presses and Dipper Approaches 
for Each Group During Extinction 


Group 

Control 
Click 
No dick 


Bar Presses 

SL7 

65.2 

72.3 


Dipper Approaches 

99E 

65.8 

60.5 


* See Coate (1056) and Rozeboom (1939) for similar 


studies with similar results. 
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were extinguished. Each bar depression produced the click during this 
phase of the experiment, and it was possible to obtain separate records 
of bar presses and dipper approaches. The results of the experiment ap- 
pear in Table 23. Quite clearly the two experimental groups produced 
fewer responses of both kinds than the control group. There is, however 
no good evidence for a difference between the t,wo experimental groups 
themselves. This indicates that the important process in latent extinction, 
at least in this situation, is the elimination of the approach response to 
food. 

CONCLUDING OBSERVATIONS 

The chief impression left by a survey of the literature on extinction is 
one of enormous complexity. A part of the explanation for this seems to 
be a great lack of parametric .data in the field. With certain notable ex- 
ceptions, the investigations cited have intended to support or refute some 
particular theoretical point ; relatively few of them have been designed to 
determine the effect upon extinction of some variable throughout a sub- 
stantial portion of its manipulate range. 

Another reason for the impression of complexity is that the variables 
controlling extinction seem to interact in ways -which limit the generality 
of statements about extinction. This occurs to a greater extent than in the 
control of learning. In certain cases these interactions create a situation 
in which the manipulation of a single variable has one effect under one 
set of conditions and the opposite effect under another. A few examples 
of this have been cited earlier in the chapter: (1) Massed trials may in- 
crease or decrease resistance to extinction, depending upon considerations 
relating to generalization decrement. (2) Resistance to extinction is 
usually greater following partial reinforcement ; blit it may be less if 
secondary reinforcement and generalization decrement are adequately 
controlled, or (3) if the partial-reinforcement schedule involved a regular 
succession of reinforced and non-reinforced trials. (4) Extinction, with 
secondary reinforcers present, is usually retarded, but it may be hastened 
if conditions are arranged to make the presence of the secondary rein- 
forcer during extinction increase the ease of making a discrimination be- 
tween training and extinction. Some additional interactive complexities 
have been demonstrated in an experiment by Ilulse (1558) who studied 
the effects upon runway performance of a number of variables among 
which were quantity and certainty of reinforcement. In extinction he 
found that the effect of the partial-reinforcement schedule depended upon 
the amount of reinforcement. Resistance to extinction following partial 
reinforcement was much greater if the amount of reinforcement was large 
than if it was small. Similarly, large amounts of reinforcement increased 
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resistance to extinction following partial reinforcement, but decreased it 

following continuous reinforcement. 

A second impressive suggestion in the literature on extinction is tliat 
extinction is not a unique phenomenon. Instead it is simply the result of 
the operation of factors which may have exactly the same influence dur- 
ing reinforced practice. Inhibition, frustration, alterations in the stim- 
ulus situation, interfering responses, and changes in secondary rein- 
forcement are all conditions which, in this chapter or elsewhere, have 
been treated as operating during learning as well as in extinction. This 
is not to say that the principles of learning as they are now understood 
are capable of handling the facts of extinction. But it seems a mistake to 
ignore variables known to be important in learning, when it is perfectly 
obvious that extinction involves the manipulation of a number of them. 

Adopting this line of reasoning inevitably means that it is no longer 
possible to hold a simple one-factor theory of extinction. In general, in- 
vestigators in the field seem to have accepted this conclusion. With a few 
exceptions, there are no longer any proponents of the classical inhibition, 
expectancy, or interference theories of extinction. This has come about in 
two ways. As in the field of learning, there have been theoretical conces- 
sions on both sides with the result that the lines separating rival theories 
have become blurred. For example, S-R theory has assimilated latent ex- 
tinction with relative ease, reducing it to the extinction of secondary rein- 
forcement and sometimes specifically evoking the r G » s G mechanism 

as a part of the explanation. Beyond this, theorists of all schools have 
tended to increase the number of principles used to explain extinction. 
Individuals sympathetic to a Hullian position have incorporated the con- 
cepts of frustration and generalization decrement into their theories ; 
and at least some expectancy theorists have admitted the possible useful- 
ness of the concept of reactive inhibition (Robinson and Capaldi, 1958). 


SUMMARY 

The term extinction refers to the gradual weakening and eventual dis- 
appearance of the CR which occurs if the CS is repeatedly presented 
without reinforcement. Since conditioned responses show little or no 
tendency to be forgotten with the mere passage of time, it is evident that 
extinction must be the result of some active process associated with non- 
reinforcement. The nature of this process is a matter upon which there 
are many very different opinions. Hypotheses which have been advanced 
to explain extinction have employed concepts as varied as those of inhibi- 
tion, response competition, discrimination, frustration, and generalization 
decrement. The first two of these are older views of extinction. The others 

mve appeared as it has become increasingly clear that the classical 
theories are inadequate. 
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Inhibition Theory 

The inhibition hypothesis maintains that every response elicited dar- 
ing extinction adds to the strength of an inhibitory tendency which op- 
poses that response. Furthermore it is assumed that inhibition increases 
with the number of responses and the effortfulness of the response, and 
that it dissipates or decays with rest. Experimental predictions to which 
this theory leads are the following: (1) Extinction will occur more 
rapidly with massed than with distributed evtmptinn I5aE ! (27 TEerafe 
at which extinction progresses will vary directly with the effortfulness of 
ih£_ies£onse. (3) Spontaneous-recovery will occur as .inhibition dissipate s 
with rest. Experimental evidence supports these predictions in a general 
way, but it also shows that some of the details of the evidence are not as 
inhibition theoiy requires; (1) Sometimes widely spaced extinction trials 
produce faster extinction than massed extinction trials. (2) Spontaneous 
recovery seldom restores response strength to the level it had attained 
prior to extinction, (3) Resistance to extinction is greater following par- 
tial reinforcement than following continuous reinforcement. These facts 
are sufficient to show that the explanation of extinction will require as- 
sumptions in addition to those made by inhibition theory. 


Generalization Decrement 

The first additional assumption adopted by the inhibition theorists was 
that of generalization decrement. If it is recognized that the conditioned 
response is, to some degree, controlled by stimuli provided hv the mem 
f act of being in the experimental situation and by traces of the reward s 
an d t>»P responses whi6h occurred on previous trials, it becomes clear t hat 
changing any of these stimuli would lead to a loss of response strength. 
For example, altering the time between trials in extinction should lead 
to relatively more rapid extinction than would be obtained Had the condi- 
tions of distribution used in training been maintained. Experiments per- 
formed to test this prediction ha ve supported it. Other evidence from 
"investigations carried out in connection with partial reinforcement and 
Cfirmtn nenus regression i ndicate that the concept of generalization decre- 
ment is of considerable” value in the understanding of extinction. As was 
true of inhibition theory, however, it does not provide a complete ex- 
planation of extinction. 


Interference Theory 

The idea which is basic to the interference theory of extinction is that 
„ f response competition- Extinction occurs, according to 1 Jw tjieoi V,' 
anew and" incompatible response ft conditional to flic CS. Although 
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denee for such an interpretation lias existed for a long time, the diffi- 
culties with interference theory hare always been very great. Some of 
these difficulties are the following: (1) The origin of the interfering 
response requires explanation. (2) Even if the source of the interfering 
response were known, it would still he difficult to account for the strength - 
ening of this response durin g nonreinforced extinction trials . (3) The 
effects of t he distribution of practice, and the phenomenon o f spontaneous 
recovery , are difficult to explain by the interference hypothesis. Facts 
like these led Hull to develop a two-factor explanation of extinction which 
combined inhibition theory with a very limited form of interference 
theory. 

According to Hull’s theory unreinforced practice results in the de- 
velopment of reactive inhibition which is a negative drive capable of 
instigating the response of resting. Resting, however, leads to the dissipa- 
tion of reactive inhibition. The reduction of inhibition is reinforcing 
(drive reducing ) with the resufTthat a conditioned resting response~rs 
acquired during extinction. This conditioned resting response (condi- 
tioned inhibition) interferes with the CR and produces a permanent 
extinction. 

The Frustration Hypothesis 

Although Hull’s two-factor theory contains many defects, the strategy 
which Hull adopted in the development of that theory seems very promis- 
ing. Tn more general terms the idea is that it may be fruitful to look for 
other motives and sources of reward in the extinction situation which 
may instigate and reinforce interfering responses. In the case of avoid- 
ance learning, the acquired fear established during learning may function 
in this way. In situations where extinction occurs because of the omi ssion 
o f a nositive reinforcer, the frustration hypothesis is applicable. 

The basic assumption in the frustration hypothesis is that the omission 
of a positive reinforcer is frustrating. Frustration is conceived as a drive 
which (a) .energizes behavior and (6) produces interfering' responses" 
which might interfere with the conditioned response. Experiments on 
frustration have provided evidence that frustration is a drive and that 
the responses produced by this drive determine the resistance of a re- 
sponse to extinction. Among the accomplishments of a frustration theory, 
which takes account of specific responses, is an explanation of the great 
resistance to extinction of habits established under condi tions of partial 
reinforcement " 1 — _ 


Expectancy Theory 

The expectancy theorist’s explanation of extinction resembles inter- 
ference theory m th a tjt involves new Jearning. In expectancy theory, 
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however, the assumption is th at extinction occurs wh en the organism 
develops an expectancy that reinforcement will no longer follow th e CS 
_or the occurrence of the instrumental response. As w ith the expectancy" 
theorist 's treatment of learning in general , perceptual variables occupy 
a position of central importance in this theory. This emphasis lias led to 
the specific hypothesis, the discrimination hypothes is, to explain the 
major phenomena of extinction. The essential point in this hypothesis is 
th at conditions which make it difficult for the orga nism to recognize that 
training has e nded and extinction began wjlllead to great resistance to 
e xtinction. _ Among the conditions leading in this direction ar<T partial 
reinforcement and irregularly patterned reinforcement. Changes'TnThe 


nning of extinction should hasten extine- 





rapidl y than those in the control grou p. The expectancy theorist 
fltt r tKiitPs~tiiis res ult to negative expectancies developed through the e x- 
ience with the empty goal box. S-R theorists have explained the fact 


?i?nf extinctio n in terms of the extinction of secondary remioreemea 








Generalization 


"When an organism lias been conditioned to respond to a particular stim- 
ulus it can be sliown that other similar stimuli will also elicit the re- 
sponse, even though these other stimuli have not been used in training. 
This ability of different stimuli to evoke a conditioned response is known 
as stimulus generalization. There is also a comparable phenomenon on the 
response side which is called response generalization. A stimulus which 
has come, through training, to elicit a particular response may, under 
some circumstances, elicit a related response without special training. It 
is evident that generalization is important in the adaptive economy of the 
organism because environmental situations never recur in nature without 
at least some changes. If situations had to be repeated in absolutely 
identical form in order for previous experience to be effective, learned 
adjustments would be of little service. 

The basic facts of stimulus and response generalization are not limited 
to conditioned reactions, but are true of unconditioned reflexes and of 
complex voluntary behavior. Every response is elicitable, not just by one 
stimulus, but by a class of similar stimuli. Correspondingly, every stim- 
ulus elicits, not just one response, but one of a class of responses. In the 
ease of unlearned reflexes, the degree of generalization is determined by 
the inherited structural organization of the nervous system. With learned 
responses, however, the degree of generalization can be modified by ex- 
perience. The study of the generalization of conditioned responses pro- 
vides the opportunity of determining the fundamental principles of such 
modification in simple, experimentally controlled situations. 


DEMONSTRATIONS OF STIMULUS GENERALIZATION 

A bi ief summary of some of the studies demonstrating stimulus gener- 
alization appears in Table 24. These experiments are ones which were 
designed merely to show the existence of the phenomenon. Investigations 
involving other parameters or attempts to establish some particular view 
of the nature of generalization have, for the most part, been omitted. 
Even these limited materials, however, suggest certain important points 
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TABLE 24 

Studies Demonstrating Generalization 

Subjects, Response, , 

Experimenter Method Stimuli Results 

Anrep (1923) Dogs, classically con- Tactile stimuli in Demonstrated gener- 
ditioned salivation. different locations on alization. Amount of 
dog’s body. salivation decreased 

with distance of test 
stimulus from CS. 

Noble (1950) Human, GSR, classi- Nonsense syllables, Demonstrated gener- 
cal conditioning. volvap (CS), nostaw alization to subvocal 
(generalization stim- stimulus, 
ulus) spoken silently 
by the subject. 

Hov/and (IftSTa) Human, GSR, classi- Pitch. 4 tones sepa- Demonstrated gener- 
cal conditioning. rated by 25 j.n.cL’s alization for condi- 
and equated for tioning and extinc- 
loudness. tion. Decelerated de- 

creasing function. 

Dittman (1949) Human, GSR, classi- Same as Hovland, Demonstrated gener- 
cal conditioning. above. alization. Function 

flatter than Hov- 
land 's. 


Philip (1947) Human, judgment of 11 cards with dif- Demonstrated gener- 
“color mass”. ferent proportions of alization. 

green and blue dots. 


Grandine and Monkeys, Wisconsin Height and bright- Demonstrated genet- 
Harlow (1948) General test appara- ness; 5 values of alization; function 
tus (see p. 386). each. Training stim- not different from a 
ulus paired with all Straight line; trend 
for test. 4 steps of (not significant) for 
increasing difference steeper generallza- 
from training point, tion following non- 
rcinforccmcnt. 


Brown (1942) Rats, non-reinforced Brightness. Three Generalization dem- 
rvnmag response to lights. 002, 5, and onstr.ited. Steeper 
food. Measured in 5,000 foot candles, gradient for higher 
terns of latency and Extremes used as intensity. Gradient 
strength of pull. training stimuli for steepened daring ex- 
different groups. tinction. 


Selilosberg and Rats. Latency of Brightness. 5 pairs Mean log latency a 

Solomon choice response fol- of stimuli, of which linear increasing 

(1943) lowing discrimination previously negatire function of bright- 

(Experiment I training on black- was always one: ness. 

„ n j y ) white discrimination, white-black; light 

white positive. gray-black; mid- 

gray-blaek; dark 




330 


CONDITIONING AND LEARNING 


TABLE 24 — Continued 


Experimenter 


Miller and 
Dollard (1941) 


Subjects, Response, 

Method Stimuli Results 

Imitative response in Leader’s "color” Demonstrated 
rats and children. changed white to generalization, 
black Tat; white to 
Negro leader. 


Watson and Child. Pear response Other furry objects, Demonstrated 

Rayner (1920) conditioned to rat. Santa Claus mask, generalization. 

fur coat, cotton. 


Brown, Bilo- Voluntary key press- Lights arranged on Demonstrated 

deau, and ing response in hu- a spatial dimension, generalization. 

Baron (1951) ; man subjects. 

Brown, Clarke, 
and Stein 


(1958) 


Bass (1958) Verbal "betting” re- Silhouettes of differ- Demonstrated 

spouses of human ent shades of gray. generalization, 
subjects. 


Grant and 

Schiller (1953) 

Human, GSR, 
classical. 

Height (area) of Demonstrated gener- 
rectangles. 3 points alization. Function 
above and 3 below convex upward for 
CS. lower stimuli. Por 

higher shows an in- 
version. Suggestion 
of stimulus intensity 
dynamism effect. 

Bass and 

Hull (1934) 

Human, GSR, 
classical. 

Place on body, shoul- Demonstrated gener- 
deT, back thigh, alization and gener- 
ankle. Extreme spots alization of extinc- 
CS. Tested at other tion. 
points. 

Hovland (1937b) 

Human, GSR, 
classical. 

Loudness. 4 tones Demonstrated gener- 
separated by 50 alization. Loudness 
j-n.d.’s. gradient flatter than 

pitch gradient. 

Guttman and 
Kalish (1956) 

Instrumental pecking Hues. 11 wave Demonstrated gener- 
response in pigeons, lengths varying over alization. Function 
Measure: rate of re- a range of 70 mu on essentially linear to 
sponse. either side of CS operant level. Dem- 

wavelength. 4 dif- onstration of gener- 
f erent CS ’s. alization in individ- 

ual subjects. 

Grice and 

Saltz (1950) 

Extinction of door Areas of circles. Generalization gradi- 
pushing response m Training at 20- and ents different for 
rats - 79- square-centimeter tests to larger and 

circles. Tests at 20-, smaller circles. 

32-, 50-, 63-, (one 
group only) and 79- 
square-centimeter cir- 
cles. 
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and raise issues to which we will return later. (1) Generalization has been 
demonstrated in many species and in a variety of conditioning situations, 
both classical and instrumental. No striking differences in generalization 
appear to characterize habits established in these two ways. (2) In many 
of these demonstrations, generalization gradients have appeared, with 
response strength decreasing in a regular fashion with increasing differ- 
ences between test and training stimuli. (3) Generalization gradients 
have been demonstrated for individual subjects (Guttman and Kalish, 
1956), establishing the reality of the phenomenon as a process charac- 
teristic of individual organisms. At one time there was good reason to 
question this point (Razran, 1949). (4) The form which generalization 
gradients most often have taken has been of the concave upward variety. 
Gradients of other forms, however, have been obtained. Such variables as 
the schedule of reinforcement during training, number of previous gener- 
alization tests, the dimension involved, and individual differences have 
all been shown to influence the shape of the gradient. Consequently state- 
ments of the exact form of some hypothetical universal gradient are un- 
warranted or, at best, premature. (5) The generalization of extinction 
has also been demonstrated. This fact has made it possible to construct a 
theory of discrimination learning which has been very influential. This is 
the theory developed by Spence, in which the concepts of the generalize, 
tion of excitation and inhibition are basic. We will describe this theory 
in the next chapter. 

Methods of Investigation 

Hovland's (1937a) study of the generalization of the conditioned gal- 
vanic skin response along a pitch dimension provides a reference experi- 
ment, In this investigation, four tones were selected by psychophysical 
methods as separated in pitch by 25 just noticeable differences (j n d/s). 

The frequencies chosen were 153, 468, 1000, and 1967 cycles. The tones, 
after being equated for loudness, represented a scale of approximately 
equal units on a dimension of pitch similarity, as judged by the verbal 
reports of subjects. One group of subjects was conditioned to the lowest 
tone and then tested, in extinction, on the others to determine the magni- 
tude of the generalized responses. A second group was conditioned to the 
highest tone and, afterwards, tested on the three lower tones. Permuta- 
tional orders of presentation made possible the pooling of the results 
obtained from all subjects into a single curve representing the generaliza- 
tion of tones separated in pitch by 25, 50, and 75 j.n.U/s. The curve, re- 
produced in Figure 11-1, shows that the degree of generalization decreases 
progressively with greater differences in pitch, producing, when plotted, 

a concave gradient. _ ...» 

This study illustrates the characteristic features of experiments on 
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FIGURE 11.1. Stimulus generalization gradient for the galvanic skin response 
conditioned to a tone of 1000 cycles per second (Hovland, 1937a). 

stimulus generalization. The typical investigation begins by conditioning 
subjects to a CS of some particular value. To test for generalization, other 
stimuli are then presented, usually in extinction. Unless separate sub- 
groups are extinguished at each stimulus value to be tested, difficulties 
arise at this point. Since the tests are carried out in extinction, and since 
the effects of extinction generalize, the measure obtained at a given stim- 
ulus point will be reduced by the effects of extinction trials carried out 
with the other stimuli. This is the reason for the permutational orders of 
stimuli during the test. They are employed in the hope that such effects 
will be balanced out and that the gradient obtained will provide a faith- 
ful representation of the underlying function. 

Other possible procedures for conducting generalization tests might 
involve the use of reinforced or non-reinf orced trials, with the test stimuli 
interspersed among the training trials with the CS. These methods intro- 
duce somewhat different difficulties. If reinforced test trials were used, 
the subjects would soon learn that they were reinforced and come to 
respond to the test stimuli in the same way as to the CS. If non-reinforced 
lest trials were used the subjects would begin to discriminate the rein- 
forced trials from the generalization tests, and this would complicate the 
measures. In addition the use of non-reinforced trials imposes a partial- 
reinforcement schedule upon the training situation, and partial reinforce- 
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ment is known to alter tbe form of the generalization gradient (see 
below). 

Tlie Generalization of Extinction 
Many of the classical studies of generalization (Bass and Hull, 1934; 
Hovland, 1937a, 1937b) included tests for the generalization of extinc- 
tion. In the experiment described above, Hovland investigated the gener- 
alization of extinction in a separate group of subjects. CR’s were first 
established in equal strength to the four frequencies. Extinction was then 
carried out for half of the subjects by presenting the highest tone without 
reinforcement. For the other half of the subjects, extinction was carried 
out with the lowest tone. The generalization of extinction was determined 
by the amount of reduction of response to the other frequencies. The 
curve of generalization of extinction was found to be similar to that for 
the generalization of conditioning, as is shown in Figure 11.2. 



FIGURE 11.2. Gradient of generalization of extinction. GSR wo* conditioned 
to all tones and then extinguished to an extreme tone. Finally tests were con- 
ducted with all tones (Hovland, 1937a). 


More recently Kling (1952) has investigated the generalization of ex- 
tinction in an instrumental situation in the following way: Groups of 
rats were trained to respond to either of two stimuli to obtain food. Tbe 
stimuli were circles of metal of different sizes. For one major group one 
stimulus had an area of 79 square centimeters and tbe other stimulus was 
*0 32, 50, or 79 square centimeters in area. For the other major group 
one stimulus had an area of 20 square centimeters and the other had an 
area of 20, 32, 50, or 79 square centimeters. It is important to remember 
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that the animals 'were reinforced for responding to both stimuli. In all, 
then there were eight groups, trained to respond to the following pairs of 
stimuli : 79-20, 79-32, 79-50, 79-79, 20-20, 20-32, 20-50, 20-79. Following 
such training, the response to the 79-square-centimeter circle was extin- 
guished for the subjects in one of the major groups. The response to the 
20-square-centimeter circle was extinguished in the other. Then the 
animals were tested on the other member of the original pair. Note that 
the more similar the two original stimuli, the more generalization of ex- 
tinction should interfere with responding in this test. The results of the 
experiment, presented in Figure 11.3, fully confirm this expectation. The 



FIGURE 11.3. The generalization of extinction of an instrumental response 

in rats (Kling, 1952). 

response measure plotted there is median latency on the test trial. The 
decreasing function is for subjects in the four groups which shared the 
small stimulus. The increasing function is for those which had had the 
large stimulus as one of the pair. Generalization of extinction is clearly 
in evidence under both sets of conditions. A question of great theoretical 
importance concerns the similarities and differences between the forms of 
the generalization gradients of learning and extinction. The early classical 
conditioning studies (Bass and Hull, 1934; Hovland, 1937a) suggested 
that the two gradients may be quite similar in terms of shape and breadth 
Kling’s results, when compared with those of Grice and Saltz (see below) 
using the same apparatus, lead to the same impression. 


VARIABLES CONTROLLING THE AMOUNT 
OF STIMULUS GENERALIZATION 

Mith the validity of the phenomenon of stimulus generalization estab- 
lished, we may turn to the question of the variables which control the 
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degree of generalization. Since these variables are almost always ones 
which are introduced during training, they characteristically affect the 
strength of response during acquisition as well as the range, slope, or 
shape of the generalization gradient. 


Generalization and the Number of Reinforcements 

With continuing reinforcement, the range of stimulus generalization 
shows an initial increase. Beyond this point, it has sometimes been found 
to show a further increase (Margolius, 1955) and sometimes to show a 
decrease (Beritov, 1924, p. 123; Hovland, 1937d). On the basis of a sum- 
mary of 67 Pavlovian experiments, Razran (1949) suggests that the 
initial increase may be followed by a decrease and then a second increase. 
The magnitudes of response strength providing the basis for this con- 
clusion are small, however, and the reality of the details of this postu- 
lated function is, by no means, certain. 

Most of the results of studies of the acquisition function for generalized 
response strength are complicated by the fact that they are presented in 
terms of relative generalization as opposed to absolute generalization. The 
distinction between these two terms is as follows : Absolute generalization 
is response strength expressed directly in terms of the measure employed 
in the experiment. Relative generalization is generalized response strength 
expressed as a percentage of response strength to the CS. The relationship 
between these measures can be made more concrete by reference to the 
results of Hovland ’s (1937d) experiment on the conditioned GSR. The 
results of that experiment, presented in these two different ways, appear 
in Figures 11.4 and 11.6. Figure 11.4 is a representation of absolute gener- 
alization. It depicts the changes in the amplitudes of reinforced and 
generalized responses as a function of practice. The upper curve shows 
the (ungeneralized) amplitude of response to the CS, and the lower curve 
depicts the changes in magnitude of the generalized responses to test 
stimuli. This way of presenting the data suggests that generalized re- 
sponse strength increases up to about IB trials and then levels off. Figure 
11.5 presents the same data in terms of relative generalization. The values 
in the lower function of Figure 11.4 have been expressed as a percentage 
of those in the upper curve. This derived function in Figure 11.5 behaves 
differently, showing an increase up to a point, and then a decrease. Func- 
tions of this form, when they have been reported, have always been for 
relative generalization, never for absolute generalization. 

Presenting these two measures together as they appear in Figures 11.4 
and 11.5 sheds some useful light on the nature of the function relating 
relative generalization to the number of reinforcements. Since the measure 
is derived from two others (the strength of the response to the CS, and 
the strength of the response to some other stiimilus used to elicit a gener- 
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FIGURE 11.4. Strength of generalized responses during the course of acquisi- 
tion. The solid line represents the amplitude of the galvanic skin response to 
the conditioned stimulus at successive stages of conditioning. The broken line 
represents the amplitude of the same response to the generalized stimulus. 

(From Hovland, 1937d.) 



FIGURE 11.5. Relative generalization as a function of number of acquisition 
trials. The data of Figure 11.4 have been replotted to show the percentage of 
generalization at successive stages of conditioning. Each point on the curve 
represents the ratio of the strength of generalized responses to the strength 
of conditioned responses. (Hovland, 1937d). 

alized response) the function in question will depend upon the shapes of 
these two more basic functions. Moreover, since the learning curves are, 
themselves, likely to be quite different, it seems most improbable that 
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different learning situations and different response measures will produce 
identical relative-generalization functions. In fact, in the absence of 
measures relating various response indices to the underlying learning 
mechanism, questions about the "true” nature of the relative-generaliza- 
tion function are unanswerable. Thus differences among these functions 
should occasion no quarrels among the individuals who obtain them. 

Hull (1952, p. 70) has proposed that changes in the form of the gener- 
alization gradient with practice depend upon the extent to which the 
response is conditioned to incidental stimuli in the learning situation. 
Such stimuli (apparatus noises, movements of the experimenter, sounds 
of the heating or cooling equipment in the room, distinctive odors, and 
so on) are often present at the time of reinforcement and should acquire 
a certain degree of potential for eliciting the conditioned response early 
in learning. Later on, these stimuli would be expected to lose their power 
to evoke the CR, through extinction, since they are apt to be present be- 
tween, as well as during, trials. Thus, early in learning, a generalization 
test should yield an increasing response strength as these nongeneralized 
incidental stimuli undergo conditioning. With further practice there 
should be a relative weakening of the generalized response as a result of 
the extinction of the same conditioned response. It will be noted that this 
pattern of acquisition and subsequent extinction is exactly what appears 
in Figure 11.5. Presumably under other conditions where the incidental 
stimuli play either a more or a less important role the course of the func- 
tion depicted in Figure 11.5 would change. 



FIGURE 11.6. Smoothed ge»e..li«.tio~ gradient. .. r.rioo. .t.ge. in ««■ 
t j on (\Vicken», Schroder, and Snide, 195*). 
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Changes in Generalization with Extinction 

Another way of changing response strength is through extinction. Thus 
alterations of the generalization gradient with non-reinforcement also 
have a hearing upon the relationship between the strength of conditioning 
and the process of generalization. Studies of this problem have, with ex- 
ceptions to he considered in the next section, found that the generaliza- 
tion function steepens during extinction (Hovland, 1937d; Brown, 1942). 
In an experiment directly concerned with this problem, Wickens, 
Schroder, and Snide (1954) conditioned the GSR of human subjects and 
tested for generalization along a pitch dimension during extinction, using 
three of the tones previously scaled by Hovland. The experimental pro- 
cedure consisted of 16 training trials followed by 8 extinction trials. 
Separate groups of subjects were extinguished at the CS and at two test 
points. Figure 11.6 presents the generalization gradients obtained on 
trials 1, 2, 3, and 8. It shows that, as extinction proceeds, the generaliza- 
tion gradient sharpens. The forms of the gradients are, however, convex 
upward, rather than concave as they were in Hovland ’s experiment (See 
Figure 11.1). 
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Guttman and Kalish (1956) trained pigeons to peck at a light of a 
certain hue, using a variable-interval reinforcement schedule to build up 
great response strength. Following this training, the birds were tested 
with the previously reinforced color and a range of others. The generaliza- 
tion gradients produced by the pigeons in successive fourths of the ex- 
tinction test appear in Figure 11.7. It will be noted that, in this case, the 
range of generalization does not decrease with extinction, but seems in- 
stead to remain about the same. As a result the gradient itself becomes 
flatter. The reason for this difference between the results obtained by 
Guttman and Kalish and those obtained by others might be due to any 
of a number of conditions. Perhaps the most likely is the variable-interval 
schedule of reinforcement employed in training. 


Individual Differences 

In the study just described, Guttman and Kalish also compared the 
generalization gradients for pigeons showing a high response rate with 
those for birds showing an intermediate or low response rate. These 
gradients appear in Figure 11.8. At least in this situation, lowered overall 
rate does not markedly decrease the range of generalization. Guttman 
and Kalish stress the fact that the effect of response strength upon gener- 
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alization seems to be much the same whether the variations are obtained 
by an extinction procedure or by selecting subjects with different rates 

of responding. 

Partial Reinforcement 

It has been known for some time that the gradient of generalization 
following intermittent reinforcement is flatter than that following con- 
tinuous reinforcement (Humphreys, 1939c). There is also reason to be- 
lieve that the schedule of reinforcement must be taken into account in 
considering the changes in the form of the generalization gradient which 
occur during extinction. In the main, the experiments reviewed so far 
seem to indicate that the range of generalization decreases as response 
strength weakens during extinction. The one study which fails to support 
this conclusion is that of Guttman and Kalish, who investigated gener- 
alization in extinction following training under a variable-interval rein- 
forcement schedule. It thus becomes a distinct possibility that the changes 
in the generalization gradient observed in extinction vary with the 
schedule of reinforcement employed during acquisition. Wickens, Schro- 
der, and Snide (1954) obtained evidence supporting this interpretation 
in the experiment described earlier. In one part of this study, they intro- 
duced a number of non-reinforced trials with a click during the condi- 
tioning of the GSR to tone. In extinction following this procedure, there 
was no tendency for the gradient to become steeper as response strength 
became weaker. This result, it will be noted, is different from that ob- 
tained following continuous reinforcement, but resembles the finding 
reported by Guttman and Kalish. 


Generalization and Motivation 

The results of experiments on the effect of motivation upon generaliza- 
tion have been consistent, showing that increased drive broadens the 
generalization gradient, both in the classical and instrumental condition- 
ing situations. In one study (Bersh, Notterman, and Schoenfeld, 1956) 
a cardiac depression was classically conditioned in human subjects by 
pairing a shock of either 20 or 28 volts with a tone of 1920 cycles per 
second. Following conditioning, the tests for generalization were carried 
out at 1920, 1020, 480, or 180 cycles per second. The results for the groups 
conditioned with high and low shock appear in Figure 11.9. Clearly the 

generalization gradient is broader following conditioning with the 
stronger shock. 

A similar effect has been obtained in experiments employing instru- 
mental procedures and using lower animals for subjects. Jenkins, Pascal, 
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FIGURE 11.9. Generalization gradients for the conditioned heart response in 
man following training with relatively weak (20 volts) and relatively strong 
(28 volts) UCS (Bersh, IVotterman, and Schoen/eld, 1956). 

and R. W. Walker (1958), for example, trained pigeons to peck at an 
illuminated circle 1.4 centimeters in diameter and tested them later, with 
this stimulus and six other larger and smaller circles which differed from 
the training stimulus in steps of 0.4 centimeters. During training and 
the subsequent test, some of the subjects were maintained at 70 per cent 
of their normal weight and others were maintained at 90 per cent. In 
order to eliminate overall differences in level of responding resulting from 
the differences in the deprivation condition, a relative generalization 
measure was used. The number of responses to each stimulus (including 
the CS) during the test period was expressed as a percentage of the 
number of responses to the CS during a previous training period of equal 
length. The results appear in Figure 11.10. Again the effect of increasing 
motivation is to increase response strength to the test stimuli. Similarly, 
Rosenbaum (1953), using human subjects, found that threat of a strong 
shock led to greater generalization of a voluntary response than threat of 
a weak shock; he also found (Rosenbaum. 193G) that highly anxious 
, • tS) f or whom the threat of shock might he very highly motivating 
flowed broader generalization gradients than Jess anxious subjects. 
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FIGURE 11.10. Effect of level of hunger upon generalization of the key-pecking 
response in pigeons (W. O. Jenkins, G. R. Pascal, and R. W. Walker, 1958). 

Stimulus Intensity 

Since increased stimulus intensity increases the vigor of at least some 
conditioned responses, the factor of stimulus intensity enters to compli- 
cate the generalization function for such responses. Intensities louver 
than the CS will lead to lessened response strength because of generaliza- 
tion decrement and the directly weakening effect of the lower intensity. 
With higher intensities, these two factors act against each other. Increas- 
ingly higher intensities will tend to strengthen responses through a 
dynamogenic effect, which Hull (1949) calls stimulus intensity dynam- 
ism, at the same time that generalization decrement tends to weaken the 
response. 

The form of the generalization function produced by variations in 
intensity will depend upon the relative contributions of these two proc- 
esses. At the very least, it is to be predicted that the generalization 
gradient for stimuli more intense than the CS will be flatter than the 
one obtained with weaker stimuli. In classical conditioning, the bulk of 
the evidence suggests that the intensity function is more important than 
the generalization function, with the result that increasing intensities 
beyond the CS produces increases in response strength with no sign of 
diminution. Razran (1949) summarizes the results of 54 Pavlovian 
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studies which manipulated stimulus intensity. A composite geaerdiza. 
tion gradient derived from these data appears in Figure 11 . 11 . Hovland 
(1937b) obtained essentially similar results in his study of intensity 



FIGURE 11.11. “Generalization” of salivary response to stimuli of intensities 
higher and lower than the CS. Data are weighted averages reported by Iiazran 
(1949) in his summary of Russian literature on generalization. 



FIGURE 31.12. Strength of conditioned GSR as a function of the intensity of 
the eliciting auditory stimnlns. Although the experiment was an investigation 
of stimulus generalization employing different intensities of tone as the gen- 
eralization stimuli, it is clear that the main effect is directly attributable 
intensity (Ilovland, 2937b). 
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generalization. Two groups, each of 16 subjects, were conditioned to 
tones of 86 and 40 decibels, respectively, and then tested to this and to 
tones of 40, 60, 74, and 86 decibels. Figure 11. 12 presents the results in 
terms of amplitude of the GSB. Again tests with stimuli more intense 
than the CS reflect only an increase in response strength. See also Hov- 
land and Biesen (1940). 

Studies using motor responses and instrumental conditioning proce- 
dures have usually obtained somewhat different results: generalization 
decrement occurs to stimuli above and below the CS, but the amount of 
such decrement is typically greater when generalization tests are carried 
out with stimuli weaker than the CS than when they involve stronger 
stimuli (W. C. Miller and Greene, 1954). Grice and Saltz (1950) trained 
rats to open a small door located in the center of a white circle to obtain 
food. For different major groups the circles were 20 or 79 square centi- 
meters in area. After training, subgroups of animals were tested in 
extinction to determine the form of the generalization functions for 
larger and smaller areas. The results of the study appear in Figure 11.13, 



in the form of generalization gradients. Note particularly the slight in 
crease in performance of the group trained on the 20 square-centime er 
stimulus when the extinction stimulus is 32 square centimeters This mav 
me resulted from the fact that the larger stimulus reflects more light 
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and is more intense; or it may simply mean that area functions as does 
an intensity dimension. 


The Effect of Changing more than One Aspect 
of the Stimulus 

The stimuli employed in conditioning experiments are always complex 
in that they at least exist on more than one dimension, and they may 
involve more than one sensory modality. Tests for generalization, how- 
ever, usually consist of altering only one carefully controlled aspect of 
the stimulus. Obviously it would be possible to change more than one 
aspect of the stimulus in order to compare the effect of such a manipula- 
tion with that produced by changing just one. Experiments of this sort 
unfortunately are rare. Pink and Patton (1953) controlled certain char- 
acteristics of the stimulus situation (visual, auditory, and tactile) in 
which rats were allowed to drink, until the animals were accustomed to 
this procedure. Then these investigators altered one or more of the 
elements of the situation under their control. They found that any 
change led to a decrement in drinking, but that changing two or three of 
the stimuli produced a progressively larger decrement. White (1958) 
taught children to pull a handle to get marbles, using a card of a certain 
hue and saturation as a discriminative stimulus. Then, for different 
groups of subjects, he changed the color of the stimulus card, its satura- 
tion, or both in a series of unreinforced test trials. Changing both dimen- 
sions produced a greater decrement than changing just one. Butter 
(1959) trained pigeons to peck at a key which carried a vertical strip 
of a certain color. In extinction tests, the color was changed, the line was 
tilted, or both of these alterations were introduced. Changes in two 
dimensions were found to produce greater decrements than changes in 
just one. The most accurate quantitative expression for response strength 
(expressed as response probability) remaining with changes in two 
dimensions simultaneously turned out to be the product of response prob- 
abilities obtained when the two changes were made independently. 


The Effect of Discrimination Training on Generalization 

The conception that discrimination will alter the form of the generali* 
ation gradient comes from a variety of sources. Hull (1953) predicts a 
steepening of the gradient in the region between the reinforced and non- 
reinforced stimuli and a decrease in the total amount of generalized 
response strength. The reason for this is that extinctive inhibition, con- 
ditioned to the non-rein forced stimulus, would generalize, reducing net 
response strength for a considerable range along the stimulus dimension. 
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This argument and the experimental evidence are developed more fully 
in the next chapter. (See pp. 364 to 369 and especially Figure 12.1.) 

Lashley and Wade (1946, p. 74) have taken an even stronger position 
declaring that “the ‘dimensions’ of a stimulus series do not exist for 
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the organism until established by differential training.” Although the 
bulk of the evidence (Grandine and Harlow, 1948; Grice, 1948) appears 
to be against this view, Jenkins and Harrison (1958) have obtained sup- 
porting data. These investigators trained pigeons to peck at a key in the 
presence of a 1000 cycles per second tone under two conditions. In non- 
differential training the tone was continuously present and the subjects 
were reinforced on a variable-interval schedule. In differential training, 
the tone was present some of the time and absent some of the time, 
reinforcement occurring only in the presence of the tone. The generaliza- 
tion gradients following these two training procedures appear in Figure 
11.14. Quite clearly, experience with the differential procedure led to 
the appearance of a gradient which non-differential training did not. 
Jenkins and Harrison suggest that other experiments, which appear to 
have produced gradients on the basis of training with nondifferential 
procedures, may have capitalized on the fact that the positive stimulus 
was typically a visual one isolated in space, so that the comparison of 
the stimulus with its background was always possible. It is to he noted 
that the changes in the generalization gradient during extinction can 
also be made to fit such an interpretation, since the subject is exposed 
to a new point on the stimulus dimension for the first time with the 
beginning of the extinction series. 


The Extinction of Generalized Responses 
Some interest centers in the ease with which generalized responses can 
be extinguished, for this bears upon the rapidity with which a discrim- 
ination can be established. Hovland (1937c) has reported data on this 
problem. After 16 presentations of a vibrator followed by a shock, when 
the generalization should be at its maximum, the conditioned galvanic 
skin response was tested both by the stimulus used in training and by 
one differing in intensity. Half of the subjects were trained with each 
stimulus, to control the effect of the difference in strength of stimuli. 
The conditioned responses to the reinforced sttmulus were found it> 
extinguish more slowly than the generalized conditioned responses to the 
other stimulus. After four extinction trials, the response to the original 
CS remained at 89 per cent of its original magnitude, while the general- 
ized response had fallen to 60 per cent of its original magnitude. The 
more rapid extinction of the generalized response was followed by greater 
spontaneous recovery, so that, 24 hours after extinction, the difference 
between generalized and conditioned responses was no longer statistically 

significant. , 

This last result is probably related to the fact that generahred re- 
sponses apparently increase in strength with time, a phenomenon which 
Bindra and Cameron (1933) describe as an incubation effect and IV™*"* 
and Wcyant (1958) call the reminiscence of generalized responses. These 
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latter investigators trained four groups of rats to run down an elevated 
runway to obtain food. Half of the rats were later tested m the same 
runway and half on a new runway which differed m color. Moreover, 
each of these groups was subdivided, half being tested immediately, halt 
tested after a week’s delay. The results of the experiment in terms of 
speed of running appear in Figure 11.15, which shows that the delay 



FIGURE 11.15. The effect of an interval between training and test upon the 
slope of the generalization gradient. Animals were trained to traverse a run- 
way of a particular color to obtain food. Following this training either by 60 
seconds or one week the animals were tested either in the training alley (T) 
or a novel alley (N) differing in color. The delay of one week leads to a 
weaker response in the tests with the training stimulus, but a stronger re- 
sponse in tests with the novel stimulus (Perkins and Weyant, 1958). 

produced a loss in response strength when the test was in the training 
runway. When it was in the new runway, however, there was an increase 
in strength. 


The Summation of Generalized Responses 

One of the important theoretical points associated with the concept of 
generalization is that it solves what Hull (1943) called the stimulus- 
learning and stimulus-evocation paradoxes. These closely related prob- 
lems both stem from the fact that stimulus situations in learning are 
probably never repeated in exactly identical form from trial to trial. 

here are always differences, produced by equipment variation or 
changes m the subject’s posture or attention. This fact poses two basic 
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problems for any strict formulation of stimulus-response theory: (1) If 
more than one stimulus-response sequence is needed for learning to 
occur, how is it possible for learning to take place, since the stimuli 
differ from trial to trial? This is the stimulus-learning paradox. (2) 
Assuming that this problem is solved, for example, by the postulation of 
one-trial learning, how could this learning ever be made manifest, again 
because the stimulus situation would never reappear exactly? This is 
the stimulus-evocation paradox. 

To handle these two problems, Hull assumed that habit strengths, in- 
cluding generalized habit strengths, summated. A graphical representa- 
tion of this hypothesis appears in Figure 13.16. Experimental tests of 



FIGURE 11.16. Graphic representation of Hull's hypothesis of the summation 
of generalized habit strength (Hull, 1943). The idea is that, although single 
reinforcements produce habit strengths of subliminal value, which are too 
weak to mediate a response, the summation of these response strengths 
leads to the development of supra threshold habit strength. 

Hull’s summation hypothesis have been only partially confirmatory. 
Bilodeau, Brown, and Meryman (1936) obtained a small amount of sum- 
mation in an experiment involving the generalization of a voluntary 
response along a size dimension. Kalisb and Gnttmnn (1957) obtained 
similar results with pigeons trained to peck at lights of two different 
colors Generalization gradients following such conditioning were ele- 
vated in the region between the two stimuli, but not elsewhere (Figure 
11.17). It seems clear that the amount of summation is considerably less 
than Hull’s theory assumes. 
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FIGURE 11.17. Empirical data on the summation of generalized habit strength. 
Pigeons were trained to peck at the two OS’s indicated in each graph. Then 
they were tested on a wide range of hues. Assumed gradients centered about 
the training stimuli are indicated by the dashed lines. Derived theoretical 
functions are indicated by the solid circles connected by continuous lines. 
Actual empirical points are depicted by open circles with black points in 
their centers (Kalish and Guttman, 1957). 

THE PROBLEMS OF STIMULUS EQUIVALENCE 
AND SIMILARITY 

One way of interpreting the results of stimulus-generalization experi- 
ments is in terms of the hypothesis that the degree of generalization 
between two stimuli is a measure of the extent to which they are per- 
ceived as similar. This is an attractive conception because of the implied 
possibility of building, at this point, a connecting bridge between the 
fields of learning and perception. Such an idea seems, directly or in- 
directly, to have been at the base of a certain amount of research on 
generalization. For example, we perceive pitches separated by an octave 
as similar and might expect responses conditioned to a certain tone to 
generalize quite strongly to tones bearing an octave relationship to the 
tone employed in training. The evidence supports this expectation. 
Humphreys (1939c) established a conditioned galvanic skin response to 
a tone of 196 1 cj cles and tested the magnitude of the generalized re- 
sponse to other tones. The response to a tone of 984 cycles, which is an 
octave below the reinforced tone, was reliably greater than that to a 
tone of 1,000 cycles, which is more similar to the reinforced tone in terms 
of physical frequency. Similarly, Blackwell, and Schlosberg (1943) 
trained rats to respond to a 10,000-eycle tone by crossing an uncharged 
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grid to obtain food. Responses to other tones (8, 7, 5, and 3 kilocycles) were 
punished and partially extinguished by means of a mild shock admin- 
istered through the grid and omitting food reinforcement. During this 
period, the 5,000-eycle tone, an octave below the reinforced tone, elicited 
responses more regularly than any tone except the reinforced one. 
Percentage of responses are plotted in Figure 11.18. The hump in the 



FIGURE 11.18. A demonstration of octave generalization in the rat. See text 
for a description of the experiment and an interpretation of this figure (Black- 
well and Schlosberg, 1943). 

curve at 5,000 cycles is the evidence for octave generalization. The 
important point is that the phenomenon of octave generalization sup- 
ports the hypothesis that generalization and the perception of similarity 
are parallel processes. 

Classical psychophysical studies of perception tend to center their 
interest, not in the perception of similarity, but in the perception of 
differences. With this minor difference, the procedures in certain psycho- 
physical experiments have a pronounced similarity to those in generaliza- 
tion Stimuli differing by certain amounts are prevented to subjects to do* 
terrains whether they ore responded to as the same or ns differ™!. The 
bosie product s-ielded by sueli investigations is a dr/erenee thrfjhoW, 
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representing, in physical terms, the smallest difference ■which can be 
perceived with reliability. 

The theory that generalization reflects the extent to which stimuli are 
perceived as similar leads to the prediction that generalization gradients 
will be related to the size of the difference threshold : Where the differ- 
ence threshold is large, the gradient should be relatively flat. Where the 
difference threshold is small, the gradient should be steeper. Guttman 
and Kalish (1956) (in the study previously described) tested this pre- 
diction for generalization along a hue dimension in pigeons, training the 
animals to peck at lights of certain colors and testing them in extinction 
at other colors. Figure 11.19 presented the results of their experiment, 
together with a graph relating the difference threshold to wave length. 
Careful examination of Figure 11.19 will show that there is very little 
correspondence between the two functions. 

This fact means either that the theory connecting generalization and 
perceived similarity is wrong, or that the pigeons’ difference thresholds 
are actually much smaller than the bottom half of Figure 11.18 suggests. 
There is some reason to believe that the second alternative is the correct 
one. Kalish (1958) performed a second study, with human subjects, and 
obtained a remarkably good correspondence between discriminability 
and generalization. The conclusion suggested by the Kalish experiment 
is that, at least in certain cases, the generalization function is the result 
of a failure to discriminate. Several other investigators (Lash ley and 
Wade, 1946; Sehlosberg and Solomon, 1943; Philip, 3947) hare also 
proposed such a theory to account for stimulus generalization. Others 
have developed theories which merely suggest that discriminability is a 
factor in stimulus generalization. These theories (Hull, 1943; Shepard, 
1957 1958) of which Hull’s is the best known, propose that generalized 
response strength is some nonlinear (for example, exponential) function 
of stimulus differences expressed in units of discriminability (j.n.d.’s). 

The use of discrimination units suggests that the perception of similarity 
is involved in discrimination. The use of a nonlinear function indicates 
that additional processes are involved. 

As a way of approaching the obvious next question, concerning the 
nature of these additional processes, it may be well to consider first some 
of the experiments in which generalization has occurred in spite of the 
fact that confusion of stimuli seems, on common sense grounds, to have 
been extremely improbable. Some of these demonstrations involve the 
classical studies of generalization. Hovland (3937b) obtained almost 
complete generalization with test tones 150 j.n.d.’s a way from the train- 
ing tone in loudness. In the Bass and Hull (1934) experiment responses 
conditioned to stimulation of the shoulder generalized to stimuli npph«! 
to the calf Brown, Bilodeau, and Baron (3933) observed generalization 
of a voluntary response trained to one light to other lights several inches 
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a - sva y. In none of these cases does it seem very likely that the subjects 

were ever confused as to the identity of the stimuli. 

Certain other demonstrations are even harder to force into a failure-ot- 
discrimination schema. Lundholm (1928, 1932) showed that it was 
possible to use hallucinated stimuli in hypnosis as a basis for condition- 
ing. Responses produced in the presence of hallucinated lights were given 
as°CR’s to genuine lights. In quite a different way, Razran (1949) pre- 
sents data which contribute to this point. Describing Russian studies of 
cross-modal generalization, he reports that a response conditioned to 
(say) a light transfers, with a strength of about 40 per cent on the 
average, to auditory or tactile stimuli. Again, the proposition that the 
subjects confused the stimuli is hard to accept. 


Semantic Generalization 

Another set of materials which argue strongly against a failure-of- 
discrimination explanation is to be found in studies of semantic generali- 
zation in which the meaning of a stimulus has been shown to provide 
the basis for generalization (Osgood, 1953, pp. 701-705). For example, 
a response conditioned to a blue light generalizes to the word BLUE. In 
other studies of semantic generalization both CS and test stimuli have 
been words. Diven (1936) performed an experiment, which, though weak 
on methodological grounds, suggested that a GSR conditioned to the 
word BARN generalized to other rural words. This result has since been 
verified by J. D. Lacey and his associates. In these latter experiments 
(Lacey and Smith, 1954; Lacey, Smith, and Green, 1955), subjects were 
required to free associate for 15 seconds to each word of a list containing, 
among others, the words COW and PAPER. For two different groups, 
a strong electric shock followed one or the other of these words. The 
critical words appeared six times in the list, providing six conditioning 
trials. The list also contained eight other words with rural connotations 
(PLOW, CORN, TRACTOR, and so on). The response measure was an 
autonomic lability score (Lacey, 1956) based upon heart rate. It was 
shown that conditioning occurred to the CS and, as Diven had found, that 
there was generalization to other rural words, mediated presumably by 
some process which equated the rural words for meaning. 

Since language habits are involved in semantic generalization, it is to 
be expected that the process will be enormously complicated. The evi- 
dence fully confirms this expectation. Semantic generalization has been 
demonstrated m experiments employing a wide range of semantic rela- 
tmnslnps, including the species-genus relationship (Goodwin, Long, and 
^ ^ 40) * and _ Synonyms, antonyms, and homophones (Razran, 

!i C , ' rTTv vi e 15 SOme evidence that generalization is most 
readily mediated by different relationships at different ages. Reiss (1946) 
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Hsing a conditioned GSR procedure and words (for example, RIGHT) as 
the CS, found that for 7- to 9-year-olds generalization was greatest to 
homophones (WRITE) ; for 10- to 12-year-olds generalization was best 
to antonyms (WRONG) ; and for 14- to 20-year-olds it was best to syno- 
nyms (CORRECT). 


The Mediation Hypothesis 

The foregoing materials indicate that, although some generalization 
apparently occurs as the result of a failure to discriminate, there are 
many cases in which other explanations must be invoked. One of the 
most fruitful hypotheses available to account for such generalization is 
through an application of Guthrie’s (1935) conception, which Osgood 
(1953) refers to as a mediation hypothesis. Specifically Guthrie’s idea 
is that all conditioning, and therefore all generalization, is mediated by 
responses of looking at, listening for, or paying attention to stimuli. 
Diagrammatically : 

S » r > s -» R. 

S, the training stimulus, produces a characteristic observing response (r) 
with proprioceptive consequences (s). In this interpretation, s plays the 
crucial role in the elicitation of behavior (R). It, rather than S, is the 
true conditioned stimulus. Stimuli (S') which differ from S may elicit 
different responses (r') with different proprioceptive consequences (s'). 
The extent to which s and s' differ will theoretically depend upon the 
difference between S and S'. If one assumes that response strength 
diminishes directly with the difference between s and s', this provides an 
explanation of the generalization gradient. In Guthrie’s view all generali- 
zation is mediated, in the sense that S only initiates n sequence of events 
which includes the true stimulus to which responses arc conditioned. 

Others, notably Dollard and Miller (1950) and Goss (1935), have 
extended such a view to situations in which the physical similarity 
among stimuli is slight. Suppose that two different stimuli (S x Sn) 
evoke the same initial response. It might then be that these two stimuli, 
however different, would be responded to in the same way, or that be- 
havior learned to one would transfer (generalize) to the other. Again, 
diagrammatically : 

If: S x r t > Si ► n A 

And ; Sr * rj *-► Si 

It follows that: Su —* ri ► si * Ra 

It will be noted that this is exactly the paradigm employed previously 
(p. 217) in connection with sensory preconditioning. It should nho be 



356 


CONDITIONING AND LEARNING 


noted that, if the initial responses to S A and S B are different (however 
similar S A and S B may be physically) the final response to the two 
stimuli should be different. Thus, in addition to a mechanism for describ- 
ing the mediated equivalence of cues, there is a similar one which wi 
deal with mediated distinctiveness. We will develop this latter mechanism 
further in the next chapter. 

The most direct implication of the mediation hypothesis is that the 
amount of generalization between two stimuli will increase if the sub- 
ject makes the same initial response to them. Grice and Davis (1958) 
put this implication to a test, using a conditioned eyeblink procedure. 
They used, as conditioned stimuli, three tones which differed widely in 
pitch. Only one, the middle tone, was positive and always reinforced. 
The subjects in this experiment also made a manual response to each of 
the OS’s, consisting of pushing or pulling a lever. For one of the non- 
reinforeed st imuli the lever response was the same as to the reinforced 
CS ; for the other negative stimulus it was different. The subjects made 
significantly more generalized CR’s to the non-reinforced stimulus in- 
volving the same motor response as the CS than they did to the stimulus 
requiring a different reaction. 

A somewhat different demonstration of the role of mediating reactions 
in learned stimulus equivalence is provided by an experiment on indirect 
conditioning. Following a procedure employed earlier by Shipley (1933, 
1935), Lumsdaine (1939) established conditioned eyeblinks in human 
subjects by pairing a light (CS) with a blow of a mechanical striker 
(UCS) on the cheek near the eye. Later a finger withdrawal was condi- 
tioned by combining the striker (now a CS) with shock to the finger. 
After this course of training it was found that the light evoked finger 
reactions in a large proportion of the subjects, although it had never 
been followed by shock. A diagrammatic analysis of a mediational ex- 


planation of this result appears in Figure 11.20. In this explanation, it 
is assumed that the proprioceptive consequences (s) of the eyeblink are 
common to the finger withdrawal and eyelid conditioning situation, thus 
providing mediating stimuli. It can be seriously doubted, however, that 
all learned stimulus equivalence is mediated by proprioceptive stimuli 
arising from intermediate reactions. In Lumsdaine ’s experiment (as in 
a related study by J. Miller and Cole, 1936), the finger response did not 
always wait for the eyelid response but in a few instances coincided 
with it or preceded it. This suggests the possibility that the eyelid re- 
sponse may not be an essential step in the sequence, but merely an overt 
indicator of a central “set” which is common to both the li°iit and the 
striker stimuli. Grice and Davis (1958) in the experiment mentioned 
previously also obtained some evidence that the set to respond was 
enough to mediate generalisation. The tendency in this direction was 
not, however, statistically dependable, and the result is only suggestive. 
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Stage I: Conditioning the eye Wink 
CS (light) — — 

UCS (striker on cheek) T£ UCR (blinkl- 

StageU: Conditioning finger withdrawal 
CS (striker on cheek) 

UCR (blink!- 


UCS (shock)- 




UCR (finger withdrawal) 


Stage HE: Demonstration of Indirect conditioning 
CS (lightly 


“•CB (blink)- 


k UCR (finger withdrawal) 


FIGURE 11.20. The mediational explanation of Lumsdaine's (1939) results. 
The light, without previous pairing with shock or finger withdrawal, comes 
to evoke this response because of proprioceptive stimulation produced by the 
eyeblink which had been conditioned to the light. 


Other scattered evidence on the importance of mediating responses 
comes from studies of long trace conditioning. It has already been pointed 
out that these responses probably involve mediating activity (p. 200). 
It is of particular interest that they seem to generalize very widely. 
According to Pavlov (1927, p. 113) they exhibit “a permanent and 
universal generalization of the analyzers.” And according to Evans 
(1925) the longer the latency of a CR, the greater its tendency to gen- 
eralize, Assuming that long-latency responses involve mediation, tin's 
again provides evidence for the importance of such processes in generali- 
zation. 

If one accepts this interpretation of stimulus generalization, ns medi- 
ated, certain questions naturally arise as to the nature of the organism’s 
mediating responses and the way in which such responses participate in 
the generalization process. Such considerations seem to lie at the base of 
Lashley and "Wade’s (1946) proposal that the dimensions of stimuli do 
not exist for the organism until it has had a chance to make comparisons 
among sample stimuli differing on the dimension. Razran (1919), going 
a step further, takes the position that, even with experience, dimensions 
never emerge. Instead the organism forms crude categories on some 
similarity-dissimilarity scale, with the result that generalization gradi- 
ents, for individual subjects, are usually uneven and stepwise. Data pre- 
sented by Razran in support of this argument show that many of the 
gradients in the early studies were, indeed, smooth only because the 
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results of several individuals were averaged. Mednick (1960) suggests 
that tlie organism may sometimes form ordinal scales as a result or 
experience with a stimulus series, and that generalization may be a func- 
tion of the number of units on such a scale separating the training and 
test stimuli. Such a theory leads to the prediction that the amount of 
response strength generalized to the same stimulus will vary with the 
number of units between the stimulus and the CS. Both Mednick and 
W. 0. Evans (1959) have shown that the strength of a generalized volun- 
tary response to a stimulus a given distance away from the training 
stimulus depends upon the number of test stimuli intervening. 

RESPONSE GENERALIZATION 

The principle of stimulus generalization, stated somewhat abstractly, 
is that an organism which has learned to respond in a certain way to 
stimulus S, has thereby learned also to respond in a similar way to a 
different stimulus S'. The counterpart of the response side is as follows : 
if an organism has learned to react with response R to a stimulus S, it 
has also learned thereby to react with a response R ' , which is unlike R, 
but in some respects equivalent to it. Wickens (1943, 1948) has described 
two experiments on response generalization involving conditioned finger 
withdrawal. In these studies an extensor response was first conditioned 
by pairing a tone with a mild shock to the finger. During this original 
conditioning the subject sat at the apparatus with the palm down, the 
middle finger resting on the shock-administering electrode. After condi- 
tioning, the subject turned his hand over so that the palm was now up. 
Under these circumstances most of the subjects responded to the CS with 
a flexor response, demonstrating response generalization. In addition, 
one of these experiments (Wickens, 1948) successfully demonstrated 
stimulus generalization of the generalized response. That is, the flexor 
response transferred in reduced strength to tones other than the specific 
one used as a CS. Other studies of response generalization have been 
conducted by Antonitis (1951). Arnold (1945), and by Williams (1941). 

SUMMARY 

An organism which has learned to respond to a given stimulus will 
tend to respond in the same way to stimuli which resemble the training 
stimulus. This phenomenon is called stimulus generalization. Often the 
amount of stimulus generalization decreases with increasing differences 
betv een the CS and the test stimulus. When response strength is plotted 
as a function of this difference, the result is a decreasing gradient the 
form of which is frequently concave upward. 
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Variables Controlling the Amount of Stimulus 
Generalization 

At the most general level, there is good evidence that the breadth of 
the generalization gradient increases with increases in the strength of 
the conditioned response. The evidence for this conclusion is to be found 
in the facts that: (1) increasing numbers of reinforcements broaden the 
generalization gradient (although a subsequent narrowing may occur), 
(2) the generalization gradient steepens with extinction, unless training 
was under conditions of intermittent reinforcement, (3) increased moti- 
vation increases the range of the generalization gradient, and (4) the 
generalization gradient is flatter following intermittent reinforcement 
than following continuous reinforcement. 

It has also been shown that the form of the generalization gradient 
can be influenced by the intensity of the test stimuli. If a response is 
conditioned to a particular stimulus and then tested with stimuli which 
are both weaker and stronger than the training stimulus, the amount of 
generalization will be greater for tbe stronger stimuli than it will be for 
the weaker. In experiments on the generalization of classically condi- 
tioned responses, this stimulus-intensity effect may be so powerful as to 
obscure the generalization gradient. 

Finally, the extent of generalization depends upon the conditions of 
training. If a response is established with the procedures of discrimina- 
tion learning, the generalization gradient is steeper than it is following 
training in which no discrimination is required. 


Methodological Issues 

Since stimulus generalization occurs on a dimension of stimulus simi- 
larity, a question arises as to the relationship between generalization and 
the dimensions obtained in the classical studies of psychophysics. There 
is reason to beliere that a part of what is reflected in the generalization 
gradient is the extent to which the organism distinguishes among stimuli. 
Some investigations have yielded generalization gradients which were 
essentially linear when plotted on a dimension of diseriminability. More- 
over the phenomenon of octave generalization suggests that a stimulus 
which is perceived as similar to the CS tends to evoke a relatively strong 
generalized response in spite of great physical dissimilarity. On the bash 
of such evidence, most theories of generalization include some reference 
to the diseriminability of stimuli. 

On the other hand, it is quite clear that it is necessary to give considera- 
tion to processes other than diseriminability in any complete account of 
stimulus generalization. This is evident from the fact that many demon- 
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strations of generalization are to be found in studies where the stimuli 
were so different that there is almost no probability that they could have 
been confused. Included in this category are cross-modal and semantic 
generalization, in which the CS and test stimuli do not exist on the same 
physical dimension. 

Semantic generalization is generalization which occurs on the basis 
of the meaning of stimuli. A subject conditioned to respond to the word 
BARN for example, may show a tendency to make the same response to 
other words with rural connotation, for example, TRACTOR. But the 
response does not occur to nonrural words sueh as PAPER. In generali- 
zation of this sort the suggestion is that the response requires some inter- 
vening activity on the part of the subject. On this basis such generaliza- 
tion may be referred to as mediated. 

It seems quite likely that all generalization is mediated in some degree 
and that an important, if difficult, consideration involves the extent to 
which mediation enters in different instances of generalization. The trans- 
fer of the conditioned GSR from a tone of one pitch to a tone of another 
probably requires only the minimal degree of mediating behavior in- 
volved in perceiving the tones. Semantic generalization, at the other 
extreme, would depend upon a process of mediation in a very important 
way. Finding means of controlling and specifying the amount of medi- 
ating activity in studies of stimulus generalization is an important un- 
solved problem. 

Response Generalization 

Response generalization refers to the fact that, when a certain response 
is conditioned to a particular stimulus, related responses are also auto- 
matically conditioned to the same stimulus. The process is the counter- 
part on the response side of stimulus generalization. Experimental studies 
of response generalization have been of very different sorts and, so far, 
have not gone far beyond a demonstration of the points that (a) the 
phenomenon of response generalization is real, (6) there seems to be a 
gradient of response generalization resembling that which occurs in the 
case of stimulus generalization, and (c) it is entirely possible for stimu- 
lus and response generalization to occur at the same time. 
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Although the tendency for a response to generalize has adaptive value 
in many situations, there are some circumstances which present an 
obvious necessity for inhibiting this tendency. One does not with im- 
punity respond to all men in uniform, all aces in a card deck, or all 
mushroom-shaped fungi in the same way. As this implies, there must be 
an influence which restricts the range of generalization and restrains the 
organism from making the same response to all physically similar stimuli. 
This influence is called discrimination. 

All learning seems to involve discrimination at least to some degree. 
A response acquired in one situation does not generalize universally. A 
response conditioned to a tone, for example, usually does not appear to 
lights or even to all tones. Sometimes it may not occur if just the loca- 
tion of the original tone is changed (Girden, 1938). In cases where cross- 
modal generalization does occur the amount of it is typically small. The 
40 per cent value mentioned in the previous chapter is probably some- 
what large. Beyond this, most learning experiments require a primitive 
sort of discrimination between the situation when the signal to respond 
is present and the same situation when the signal is absent. The im- 
portance of this fact has been stressed by Mowrer and Lamorcaux (1931) 
and by C. C. Perkins (1953). 

Training of the sort just described does not result in very fine dis- 
crimination. To produce a more precise discrimination among stimuli 
it is necessary to employ some special procedure which makes reinforce- 
ment contingent upon a response to one of two or more stimuli. When 
such procedures are introduced, the differentiating power of mammalian 
organisms can be demonstrated to be remarkably great, extending to n 
wide range of stimuli. With proper training, discriminations between 
drive levels, amounts of work (Thompson, 1914), delays of reinforce- 
ment (Logan, 1932), as well as among the obvious exteroceptive stimuli, 
have been established. 


3 61 
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EXPERIMENTAL METHODS 

The general method required to produce a discrimination involves the 
extinction of generalized responses by non-reinforcement, "while the 
strength of the response to some particular stimulus is maintained by 
reinforcement. Two more specific procedures may be identified. These 
have been called the methods of successive and simultaneous presentation 
of stimuli by certain investigators ("W. S. Hunter, 1914). 

The Metliod of Successive Presentation of Stimuli 
(Contrasts) 

In Pavlov’s method of contrasts, now more often called the method of 
successive presentation of stimuli, only one of two stimuli to be discrim- 
inated is presented on each trial. Reinforcement follows one stimulus and 
not the other, with the result that the subject comes to respond to one 
of the two stimuli and to inhibit the response to the other. As originally 
conceived, the method was applied within the framework of the classical 
conditioning procedure and used to establish discriminations between 
different stimuli in the same modality, such as two tones of different 
pitches. It is also the technique basic to the establishment of conditioned 
inhibition in which a CS is reinforced if presented alone but not if 
presented with another (conditioned inhibitor). Similarly, the demon- 
stration of patterning involves the method of successive presentation. In 
this procedure there are two stimuli. Presented together, they are rein- 
forced, but either of them presented alone is not (Hull, 1940, 1945a). 

Experiments on discrimination of the instrumental conditioning vari- 
ety, using the method of successive presentation, have been carried out 
by Elder (1934, 1935), Wendt (1934), Yerplank (1942), and by 
Antoinetti (1950). Elder taught chimpanzees to react to a telegraph 
key following a ready signal, provided a tone was present. The animals 
learned to refrain from response after the ready signal when the tone 
was absent. In Wendt’s study, monkeys learned to open a drawer which 
contained food following the positive auditory stimulus. The drawer was 
locked following the negative (that is, absent) stimulus, and the animal 
learned to withhold reaction. Antoinetti ’s experiment, in which rats were 
used as subjects, was similar in conception. First the animals learned to 
open a door containing a black card to get out of a starting box and 
obtain food. Following this initial training, discrimination learning 
began. The black card continued to be followed by reinforcement, but 
on randomly alternated trials a white card was substituted and this card 
was not reinforced. As a result of this training a discrimination gradu- 
ally developed. The rats responded very quickly to the black door but 
they responded slowly or refused to leave the starting box on trials with 
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the white door. Verplank's procedure was similar to Antoinettes, except 
that the apparatus was the Graham-Gagne runway and reinforced and 
non-reinforced trials were alternated according to the regular schedule. 

Skinnerian methods of discrimination training are also modified ver- 
sions of the method of successive presentation of stimuli. Responses in 
the presence of a positive discriminative stimulus (S D ) are reinforced; 
and responses in the presence of a negative discriminative stimulus (S 4 )’ 
which is usually the absence of S D , are not reinforced. With training,’ 
high rates of responding are established to S D and very low ones to S 4 
(if, H. Smith and Hoy, 1954). 


The Method of Simultaneous Presentation 
(Choice Reaction) 

In instrumental learning situations, the development of discriminations 
is more commonly studied in experiments where the positive and nega- 
tive stimuli are present simultaneously. The simplest of these is the 
position discrimination required in simple T- or Y-mazes in which the 
animal is reinforced for going to one side or the other. In somewhat 
more complicated experiments using other stimuli, the apparatus is very 
often an adaptation of the T- or Y-maze; but the positive stimulus, which 
may be a card of a certain color or pattern, a light, a floor of distinctive 
texture, or an alley of a certain width, is shifted randomly from side to 
side on different trials. If the subject responds to this stimulus, it is 
rewarded. If it does not respond correctly there is no reward and punish- 
ment may be administered. Using adaptations of this basic plan, many 
different kinds of apparatus and reactions have been used in studies of 
discrimination learning. Subjects have been trained to pull a string on 
the left or right, which will bring food if correct (Kluver, 1933), to open 
one of two possible boxes to secure food (Spence, 1934), to jump through 
an opening (Lashley, 1930b), to push open a door (Munn, 1931), and 
so forth. 

There are two important differences between the successive and simul- 
taneous procedures for studying discrimination learning. (1) Since the 
two stimuli are presented simultaneous!}' in the latter procedure, the 
relational characteristics of the stimuli may be easier for the subject to 
notice than when the stimuli are present separately. (2) In simultaneous 
discrimination the choice is usually between two reactions (for example, 
turning right or left) rather than between reaction and restraint. Each 
of the reactions, thus, may have a double determination, being an ap- 
proach to one of the stimuli and at the same time a reaction away from 
the other (Nissen, 1931). The first of these differences has been most 
stressed in theoretical discussions of di-crimination learning. Theorists 
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who hold that discrimination involves a response to a relationship have 
argned that the method of simultaneous presentation should lead to 
quicker learning than the method of successive presentation. Empirical 
studies on this point will be reviewed in a later section. 

DISCRIMINATION AS COMBINED CONDITIONING 
AND EXTINCTION 

Looked at in operational terms, one fact about discrimination learn- 
ing is inescapable: The procedure is a combination of two simpler proce- 
dures. Trials with the positive, or reinforced, stimulus (S+) are condi- 
tioning trials-, trials with the negative, or non-reinforced, stimulus (S— ) 
are extinction trials. Given this fact it is not surprising that several 
theorists have proposed explanations of discrimination learning which 
involve the concepts of conditioning and extinction. Among such condi- 
tioning-extinction theories, the most important are those of Spence 
(1936, 1937a, 1937b) and Hull (1939b, 1943, 1950, 1952). Conditioning- 
extinction theory, as originally developed by Spence and Hull, entails 
the following assumptions: (1) that every reinforcement leads to an 
increment in the (excitatory) tendency to repeat the reinforced response, 
(2) that every non-reinforcement leads to an increment in the (inhibi- 
tory) tendency not to respond, (3) that both of these tendencies general- 
ize to other stimuli, (4) that the magnitude of the inhibitory tendency 
is less than that of the excitatory tendency (otherwise learning could not 
occur in any partial-reinforcement situation, S-j- and S — being the same 
stimulus, where reinforced and non-reinforced trials occur in a ratio of 
1:1 or less), (5) that the excitatory and inhibitory tendencies interact 
algebraically; and (6) that discriminatory reactions are based on the 
resolution of the competing tendencies in favor of the reaction to the 
stimulus which has the stronger tendency conditioned (or generalized) 
to it. In experiments involving the method of successive presentation of 
stimuli, a discrimination is said to have been established when the sub- 
ject characteristically responds to S-f- , but does not respond to S— . The 
theoretical account explains such discriminations by the assumption that 

the processes of reinforcement and extinction have provided S-f- and S 

with habit loadings which, respectively, are predominantly positive and 
negative. In experiments involving the simultaneous presentation of 
stimuli or choice reactions, the response theoretically occurs to the 
stimulus with the stronger excitatory strength conditioned to it. 

. In ordcr t0 makc Predictions about the outcome of specific experiments, 
it is necessary to add a seventh assumption to the list above: (7) that 
the stimuli manipulated by the experimenter are in fairly direct control 
of the subject’s behavior. In situations where mediating stimulus- 
producing reactions are important, it would be necessary to know more 
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than we usually do about the functioning of such stimuli to explain the 
results of a discrimination experiment. As we shall see, studies which 
maximize the role of such responses cannot always be explained by 
conditioning-extinction theory -without modification. 

Figure 12.1 provides us with a concrete representation of Hull’s 



FIGURE 12.1. Hull’s theory of discrimination learning. The upper curve is 
a hypothetical generalization gradient established hy conditioning a response 
to the stimulus corresponding to point zero on the baseline. The lower, 
shaded, gradient represents the generalization of extinction carried out with 
a non-rein/orced stimulus 8 j.n.d.’s away from the positive stimulus on the 
baseline. The smaller, asymmetrical function represents the diflerence between 
the two other functions (Hull, 1943). 

(1943) version of the conditioning extinction position, in the form of 
three generalization gradients. The upper curve is the excitation func- 
tion generated by reinforcing a stimulus (S-}~) at the point marked 
zero on the baseline. The lower, shaded, curve is the inhibitory function 
developed as a result of non-reinforcement of a stimulus (S-— ) 8 j.n.d.’s 
away from S-f- on some stimulus dimension. The intermediate dotted 
curve is the residual obtained by subtracting the points of the inhibitory 
function from those on the excitatory one. This has been called a post- 
discrimination gradient (Hull, 1932) and represents the form which a 
generalization gradient obtained after discrimination training theoreti- 
cally should take (Passey and Herman, 3935). 

Formulated in this way, couditioning-cxtinction theory lends to a 
variety of predictions, some of which have been tested in n differential 
eyelid conditioning experiment by Ovnther (1937). In this study the 
positive and negative stimuli were small lights about two inches apart 
directly ahead of the subject. The right-hand light was followed by on 
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air puff; the left-hand light was not. The first prediction from Hull’s 
theory which Gynther set out to test involved the level of conditioning 
attainable in discrimination learning. Reference to the dotted line in 
Figure 12.1 will show that the generalization of inhibition from the 
negative stimulus to the positive one theoretically should reduce the 
level of performance to the positive stimulus. To test this deduction 
Gynther ran two groups of subjects. One was conditioned in the ordinary 
way, receiving 50 presentations of only one stimulus which was always 
followed by the air puff. The other group received the same number of 
reinforced trials with the same stimulus, but there was an equal number 
of non-reinforced trials with the negative stimulus. The results for these 
two groups appear in Figure 12.2. The upper curve is for the regularly 



FIGURE 12.2. A comparison of performance in eyelid conditioning tinder 
conditions of discrimination and straight conditioning (Gynther, 1957). 


conditioned group. The lower two curves show the responses to the posi- 
tive and negative stimuli obtained from the discrimination group. In 
correspondence with the theory, the discrimination group gave fewer 
conditioned responses to S+ than the conditioning group. Passey’s 
(195 1 ) data also support this deduction for the running response of rats 
It will also be observed in Figure 12.2 that there was a substantial 
level of responding to S— . This result has typically occurred in studies 
of differential eyelid conditioning and theoretically can happen under 
circumstances where (1) the excitatory process conditioned to S-f is 
greater than the inhibitory process conditioned to S— , and (2) the 
positive and negative stimuli are close together on the stimulus dimen- 
sion. The first of these conditions is assumed by the theory. The second 
is a matter of experimental manipulation. Other evidence in Gynther ’s 
study suggests that S+ and S- are, as these results indicate, close to- 
gether on the generalization gradient. 

Th, discrimination group in this portion o£ Gjnther’s experiment 
a “ a con,ro1 =' ro "P against which the effects of a number of other 
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procedures could be compared. One of these involved the ratio of rein- 
forced to non-reinforeed trials. To understand the result of this manipu- 
lation, suppose that the gradients of excitation and inhibition in Figure 
12.1 are those which exist after SO reinforced and 50 non-reinforced trials, 
as in the case of Gynther’s discrimination control group. Administering 
50 reinforced trials with S+ and only 17 with S- (a ratio of 3:1) would 
leave the gradient of excitation unchanged, but the inhibitor}' gradient 
would be weakened considerably. The effect of this weakening of the 
inhibitory gradient should be two-fold: (1) There should be a higher 
level of response to S-f- because of the smaller amount of inhibition 
generalizing to S+. (2) At the same time, there should also he a higher 
levei of response to S— , because the excitation which generalizes to S— 
will suffer less from inhibition. The results of the comparison implied in 
these predictions appear in Figure 12.3. It will he observed that the 



FIGURE 12.3. Performance in differential eyelid conditioning under condi- 
tions where the reinforced stimulus and unreinforced stimulus are presented 
in a ratio of 3;1 or 1:1 (control group). The data are from Gynther (1957). 

predictions were confirmed. Moreover, it appears that the increased 
response strength to S~f- and S— in the 3:1 group was about the same 
to each of these two stimuli. The degree of discrimination (responses to 
S — subtracted from responses to S-f~ ) did not differ for the two groups. 
G. N. Cantor and Spiker (1954) obtained essentially the same result in 
an experiment where preschool children learned to discriminate between 
toy cars of different colors, receiving reinforced and non-reinforced trials 
in the ratio of 2 :2 or 2 :1.* 

To in trod nee a third prediction suppose that a subject has been condi- 
tioned to respond to S-f- before discrimination training begins. In terms 

>Tvro other investigations of this problem (Birch, 1935; FiUwatcr, 1952) used the 
procedure ot keeping total number of trials constant and varying the number of re- 
inforced and non -reinforced trials within this total. This procedure makes the results 
hard to interpret in terms of Figuro 12.1, because, theoretically, both the eieltatory 
and inhibitory gradients are manipulated Jn ways which cannot lw assessed. 
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of Figure 12.1, this is to say that the gradient of excitation is established 
without setting up the inhibitory one. With the beginning of discrimina- 
tion training, inhibition develops and generalizes to S+ . The result should 
he a decrement in response strength to S-j-. To test this prediction 
Gynther gave another group of subjects 20 conditioning trials before 
initiating discrimination training. At the end of the preliminary condi- 
tioning trials, the subjects were giving approximately 60 per cent CR’s. 
During the first 10 conditioning trials the level of response to S+ 
dropped to less than 50 per cent. Similar results, usually of lesser magni- 
tude have been reported by Antoinetti (ef. Hull, 1952, p. 62), Spence 
and Beecroft (1954), and Hilgard, Campbell, and Sears (1938). 

One of the most obvious predictions from Hull’s theory is that psycho- 
physieally dissimilar stimuli will be more easily discriminated than simi- 
lar stimuli. (In Figure 12.1, this would be represented by separating the 
two gradients on the stimulus dimension). To test this prediction, 
Gynther increased the difference between stimulus lights S-j- and S — by 
covering the latter with a red filter which changed its color and reduced 
its intensity. Figure 12.4 shows the result of this procedure. Note that 



FIGURE 12.4. Performance in differential eyelid conditioning with two dif- 
ferent degrees of similarity of positive and negative stimuli (Gynther, 1957). 


the difference in response level to S+ and S— increases, and that the 
absolute level of responding to S+ also increases. Both of these findings 
are consistent with the theory. 

At the other extreme as regards degree of similarity is the partial- 
reinforcement situation in which S-f and S- are identical. In terms of 
Figure 12.1, tins would involve drawing the shaded inhibitory gradient 
as a shortened mirror image of the excitatory gradient, with the peaks 
of the two functions opposite each other on the baseline. Subtracting 
values on the inhibitory gradient from those on the excitatory gradient 
in this case would produce a residual gradient with a smaller maximum 
than could be produced by any other combination of positive and nega- 
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tive gradients. In behavioral terms, response strength should be lower 
under a partial-reinforcement schedule than under any other. A group 
run under these conditions confirmed this prediction. In their last 10 of 
100 trials, the percentage of CR’s was 39 per cent. The same values for 
the discrimination group (responses to S-}-) and the conditioning group 
were 50 per cent and 66 per cent, respectively. Obviously, and in line 
with the theory, response strength to S-f- is reduced in proportion to 
the similarity of S-f and S— . This same prediction has been tested by 
Raben (1949) in an experiment which employed rats as subjects and 
the Graham-Gagn6 runway as the apparatus. As in Gynther's study 
there was clear evidence that the ease of discrimination varied directly 
with the difference between the stimuli. 


Criticisms and Alternative Formulations 
of Conditioning-Extinction Theory 
In spite of its obvious utility in suggesting experiments and in pre- 
dicting the outcomes of certain of them, conditioning-extinction theory 
has been subjected to considerable criticism. In some cases this has led 
to a reinterpretation of the basic processes supposed to lie behind dis- 
crimination. In other cases, experiments suggested by the criticisms 
have led to a strengthening of the underlying assumptions. In still others, 
new and useful ways of looking at discrimination learning have emerged 
as an indirect result of the criticism. Let us look at some of these criti- 
cisms and problems and then observe the effect that they have had upon 
the theory. 

1. The Lashley-Wade Criticism. As a part of their attack on “neo- 
Favlovian” psychology, particularly as it uses the concept of generaliza- 
tion, Lasbley and Wade (1946) criticized the conditioning-extinction 
explanation of discrimination learning very severely both on theoretical 
and empirical grounds. As a crucial test of this theory, they proposed the 
following experiment : Two groups of rats were to be trained in n dis- 
crimination situation. One group was to be rewarded for responding to 
a 5-cm. circle as opposed to a black card. The other was to be rewarded 
for responding to an 8-cm. circle as opposed to the same black card. Then 
the two groups were to be trained to discriminate between the two circles 
with the 8-cm. circle being positive. Conditioning-extinction theory 
would predict that the animals previously trained to the 8-cm. circle 
would transfer this learning to the new discrimination situation and 
learn it more quickly than the group trained previously on the 5-cm. 
circle. Experiments reported by Lashley and Wade, however, produced 
the opposite outcome. That is, learning was more rapid by the group 
previously trnined on the 5-cra. circle. Although it is by no means clear 
that any other theory would predict the results which were obtained. 
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such data do pose obvious difficulties for the theories of Hull and Spence. 

Attempting to answer the criticisms implicit in the Lashley -Wade 
studies, Grice (1948c, 1951) performed two experiments which followed 
the Lashley- Wade plan ; hut he also made certain changes designed to 
eliminate complicating aspects of the original procedure. The most im- 
portant complications were the following: (a) Lashley and Wade had 
used p unishm ent for incorrect responses. On trials when an error oc- 
curred the rat jumped against a locked door and fell into a net. Thus, 
if the animals originally trained with the 5-cm. circle had transferred a 
tendency to respond to it to the second discrimination situation, they 
would have been punished oftener than the 8-cm. animals early in this sec- 
ond phase of training. Such punishment might, then, have led to their 
abandoning the response to the 5-em. circle very quickly, (b) Lashley and 
Wade had used the so-called correction method (Hull and Spence, 1938) 
in which a trial is counted only after the animal makes a correct response, 
no matter how many errors occur in such a trial. Grice used a non-correc- 
tion method and allowed only one correct or incorrect response per trial. 
Under these conditions Grice obtained the results appearing in Figure 
12.5, which are clearly in line with the results of conditioning-extinction 
theory in that response strength acquired in one situation was shown to 
transfer to another, benefiting the development of a discrimination. 

In his second experiment, Grice (1951) provided the same sort of 
evidence for the transfer of extinction. Two groups of rats were trained 
to respond to a black square rather than to an 8- or 5-cm. circle, respec- 



.ho some os .ho previously reinforced ..Ltlut il 'he othlr 
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lively. Then both groups were trained to discriminate the two circles 
with the 5-cm. circle positive and 8-cm. circle negative. The group 
originally trained with the 8-cm. circle negative made only 3.7 errors in 
learning the new discrimination, whereas those trained with the 5-cm. 
circle made II.7 errors. These results, again, conform to the predictions 
from conditioning-extinction theory. 

For reasons which may be attributable merely to slight procedural 
variations or to differences in strain of rats, learning in Grice’s second 
experiment was much more rapid (3.7 and 11.7 errors for the two 
groups) than it was for the first experiment (12.9 and 27.0 errors). The 
difference seems worth mentioning, however, because it is in line with 
F. K. Graham's earlier (1943) discovery that what is transferred in 
the transfer of a discrimination is chiefly a tendency not to respond, in 
conditioning-extinction terms, the extinctive or inhibitory tendency. 

2. The Problem of Motivation. One of the most obvious predictions 
from the original Spence-Hull version of conditioning-extinction theory 
is that increasing motivation will facilitate the development of a dis- 
crimination. In Hull's (1943, 1952) terms this is expressed abstractly 
by saying that drives (D) and the difference between net habit strengths 
(H) to S-f- and S— interact multiplicatively. That is : 

R s+ - = f (D) X f (H s+ - H g _) 

Clearly, increasing drive should increase the preference for I?s+ when- 
ever H s+ > Hs — Experimental evidence on this point is inconsistent. 
Some investigators (Buchwald and Yamaguchi, 1955 ; Tolrnan and Glcit- 
maa, 1949; Powlowski, 1953,- Dinsmoor, 1952; Eisman, 1956; Eisman, 
Asimow, and Maltzman, 1956) have obtained evidence that increased 
drive improves discrimination. Others have obtained either no effect 
(Teel, 1952; Teel and Webb, 1951; Miles, 1959; Meyer, 1951) or an indi- 
cation that, under certain conditions, increased drive may actually in- 
terfere (Bruner, Matter, and Papanek, 1955; Yerkes and Dodson, 1908). 
Results of the latter two kinds pose an obvious problem for conditioning- 
extinction theory. 

This difficulty could be removed if there were a reasonable basis for 
making the assumption that the experiments in which drive failed to 
benefit discrimination were ones in which lla+ and II a - were equal. 
Under these conditions, drive should theoretically have no beneficial effect 
upon discrimination. A revision of conditioning-extinction theory out- 
lined by Spence (1958) makes such an assumption tenable for certain 
forms of learning. Whereas Spence’s earlier writings had at least implied 
that the law of effect applies to all learning, he was led, in 3956. to alter 
this position and to propose a variety of two-process theory. This was ap- 
parently due, in part, to the difficulties encountered by uniproc^s rein, 
forcemcnt theory in dealing with the effects of motivation on divrhnina. 
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tion. In Spence’s revised theory, classical conditioning is assumed to obey 
tlie lav of effect, and instrumental learning is thought of as involving 
only a eontiguity principle. It vrill he noted that this theory is exactly the 
opposite of the more conventional two-process position. 

The implications of the revised theory for the outcomes of experiments 
investigating the effect of drive on discrimination differ somewhat de- 
pending upon whether the discrimination is established by classical or 
instrumental conditioning procedures. In classical differential condition- 
ing, habit strength accrues to S-f- only on reinforced trials, and S— re- 
ceives no habit strength except through generalization. Thus there . is 
always a difference in the strength of H S -u and H s _. In theory, drive 
multiplies this difference, and the straightforward prediction is that in- 
creased motivation will improve discrimination. In their total effect, 
classical diff erential conditioning experiments support this prediction 
(Runquist, Spence, and Stubbs, 1958 ; Spence and Beecroft, 1954 ; Spence 
and Farber, 1953, 1954), although there are exceptions (Hilgard, Jones, 
and Kaplan, 1951) and the differences in individual experiments have 
usually not been statistically significant. This latter fact may be the result 
of a phenomenon considered in the last chapter, where we saw that in- 
creasing motivation increases the breadth of the generalization gradient. 
This effect would reduce somewhat the difference to be multiplied by 
drive, thereby working against the prediction. 

For instrumental conditioning experiments, the situation is different 
because the law of effect, according to Spence’s new theory, does not 
operate. In such an experiment, the habit strength associated with the 
positive and negative stimuli will be equal if these stimuli have been re- 
sponded to (rewarded or not) an equal number of times in the discrimina- 
tion experiment, and drive changes should have no effect upon dis- 
criminative performance. Experiments in which drive has had no effect 
upon discrimination appear to have approximated this condition. Such 
reasoning has led Spence and his associates (Ramond, 1954; Spence, 
Goodrich, and Ross, 19o9) to perform a series of experiments in which 
an attempt was made to manipulate the relative strengths of H s + and 
Hs — simultaneously with the manipulation of drive, and to observe the 
effect of such procedures upon performance. In the latter of these studies, 
for example, rats learned a black-white discrimination under 3 or 40 hours 
of food deprivation. In one subeondition free- and forced-choice trials 
were arranged so that the numbers of responses to the positive and nega- 
tive stimuli were equal. In a second subcondition there were twice ^as 
many responses to the positive stimulus as to the negative. Speed of re- 
sponding increased with drive under both subeonditions ; but discrimina- 
tion improved only when the arrangement of trials favored the develop- 
ment of relatively stronger habit strength to S-f . Thus, at the cost of a 
major theoretical concession, conditioning-extinction theory can be made 
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to explain the results of at least some experiments which manipulated 
motivation. 

. ^ Discrimination Learning with Two Reinforced Stimuli. Studies bear- 
wg on this revised, non-reinforcement, theory of discrimination are also 
to be found in investigations where the two stimuli are both associated 
with reward, but where a less effective reward is used for the "incorrect" 
stimulus. Logan (1952) trained rats in a position discrimination with 
short and long delays of reinforcement following the two responses. In 
this study, the animals learned to press either of two bars, one on the 
right of the apparatus and one on the left, to obtain food. One response, 
however, was reinforced after 1 second ; the other was reinforced after 5 
seconds. Responses to the two bars were carefully equated through the use 
of a series of free and forced trials. Under these conditions, the rats re- 
sponded faster to the bar with the short delay and, on free-choice trials 
selected it in preference to the bar with the long delay. Reversal of the 
delay conditions led to an appropriate reversal of both of these tendencies. 
The speed results appear in Figure 12.6. In experiments of this sort, 



BLOCKS OF EIGHT TRIALS TO EACH BAR 

FIGURE 12.6. Discrimination and reversal of a discrimination In a situation 
where responses to both stimuli were reinforced. Animal* were run In an 
apparatus where touching a bar produced food; but the time between pressing 
the bar and the delivery of food differed, being one second for one bar and fisr 
seconds for the other. The graph above ihowa speed of responding to the 
short- and long-delay bars for two group* of aubjeet*. For one group the delay 
conditions were reversed after 80 trials. For the other there was no reversal 
(Logan, 1952). 
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Spence’s revised theory would require that the habit strength associated 
with the two responses be equal because of the equivalent numbers of 
trials with each response. If this interpretation is correct, the differential 
performance must have resulted from an incentive mechanism (p. 170) 
derived from a classical conditioning of responses during the delay to 
stimuli associated with the right and left sides of the apparatus. This 
suggests the intriguing possibility that, with trials equated, in instru- 
mental conditioning discriminations may be established through the classi- 
cal conditioning mechanism involved in the development of incentive 
motivation or secondary reinforcement. The implications of this idea are 
largely unexplored. 

4. The Learning-Performance Distinction in Discrimination Learning. 
As a part of the revision of his general theory of learning, Spence has 
also (1956, 19o8) brought an important aspect of the learning-perform- 
ance distinction to bear on behavior in selective learning. Specifically, 
he argues that the curves shown in Figure 12.1 (and comparable ones 
vhieh might be constructed to represent the new contiguity interpreta- 
tion) are reflections of the underlying learning process. There is good 
reason to believe that behavior in the selective learning situation will not 
mirror these functions in anything like a one-to-one fashion. Employing 
a modification of Hull’s assumption of an oscillatory process, Spence 
sometimes uses a transformation which makes the probability of choosing 
S-f m selective learning a normal integral (z-score) function of the dif- 
ference m response strengths conditioned to S-f and S- At the same 

1,8 U58) d3imS Procedut! probably 

provides but a poor first approximation to an adequate statement of the 
transformation involved. 


RELATIONAL THEORY 

Particularly in the case where the stimuli to be discriminated are pre- 
s nted simultaneously and in situations involving choice reactions an 
a ternative explanation of discrimination seems possible This is rela 
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often hold (c) that the development o{ discrimination involves, on the 
part of the organism, the adoption of a series of hypotheses -which are 
successively discarded until the correct one is hit upon. This non-con- 
tmuity position, however, is not limited to discrimination learning. It is 
a theory about the nature of learning in general and, as such, ha°s been 
discussed earlier (p. 136). 


Simultaneous and Successive Discrimination Learning 
The proposition that discrimination learning involves an act of com- 
parison suggests that such learning should proceed most rapidly under 
conditions which maximize the opportunity for such comparing behavior 
to occur. A direct test of this idea seems to be provided by the relative 
speeds of mastering simultaneous and successive discriminations. In 
simultaneous discrimination, positive and negative stimuli are presented 
together, ordinarily in some sort of random spatial arrangement from 
trial to trial. The learner’s task is to choose between the two stimuli, both 
of which are present for inspection and comparison. In successive dis- 
crimination, however, only one stimulus (or a pair of identical ones) is 
present. The stimuli differ from trial to trial, and the organism roust 
make one response in the presence of one stimulus (or pair of stimuli) 
and the other response in the presence of the other. 2 In this procedure it 
is obviously not possible for the organism to make a direct comparison of 
stimuli. Relational theory holds that a comparison actually docs occur, 
between the stimulus now present and a memory trace of the alternative 
stimulus, and that such comparison is harder to make than one with both 
stimuli physically present. This, more exactly, is the reason that relational 
theory predicts relatively greater difficulty with successive discrimina- 
tions, Absolute stimulus theory predicts no difference. 

Experimental comparisons of the two forms of discrimination learning 
have produced all possible results. In some experiments (Spence, 1952; 
North and Jeeves, 1956) simultaneous discriminations have, as predicted 
by relational theory, been easier than successive discriminations. In others 
there has been no difference (Grice, 1919). In still others the successive 
discrimination has been found to be easier (Bitterman and Wod insky, 
1953; Weise and Bitterman, 1951; Teas and Bitterman, 1952). The im- 
plications of these various outcomes for discrimination learning theory 
seem to be as follows: (1) Both of the initial theoretical alternatives 
relational theory and absolute stimulus theory, lead to erroneous predic- 
tions, indicating that both must be modified in order to explain the results 
of these studies. (2) Since opposite outcomes have been obtained, it would 
seem that the relative effectiveness of the two procedures must depend 

'Tfets proecdure differs from tho method described earlier in that it £s po»iit>te to 
moke a correct response and obtain reward on each trial. 
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upon the operation of other variables. Of such variables, one of tlie most 
important appears to be the similarity of the stimuli to be discriminated. 
In easy discriminations, there is little or no difference between the two 
procedures, but in difficult discriminations the simultaneous method is 
superior (Loess and Duncan, 1952; McCaslin, 1954). (3) In certain ex- 
periments deriving from the simultaneous-successive discrimination issue, 
another problem has been raised. This problem involves the nature of the 
stimuli controlling responses in the discrimination situation. 

This last problem is basic and has plagued students of discrimination 
learning for a long time. All theories in this area begin with the implicit 
assumption that the organism perceives the stimuli presented. If it does 
not, no theorist would expect the discrimination to be learned. This simple 
statement, however, obscures certain differences of emphasis which sepa- 
rate absolute stimulus theory and relational theory. Absolute stimulus 
theory holds that it is sufficient if the stimulus energy reaches appropriate 
receptors. Relational theory, on the other hand, assumes that the animal 
must, in addition, “pay attention” to the stimuli in question, a concep- 
tion which modern physiological research makes less mysterious than it 
once was (Lindsley, 1957). 


In experimental terms, the absolute stimulus theory seems to say that 
all stimuli impinging upon the receptors of the organism should control 
the lesponse in some measure. Some relational theorists have extended 
this conception, perhaps unfairly, and have attempted to put the absolute 
stimulus theorists in the position of saying that small incidental differ- 


ences between two stimuli will be discriminated, even if reinforcement is 
in no v ay contingent upon a response to one of these stimuli as opposed 
to the other. To illustrate: Teas and Bitterman (1952, Experiment II) 
trained two groups of rats in the following way : Original training for 
both groups consisted of two successive discriminations in a Lashley lump- 
ing apparatus, as outlined in Table 25. In this phase of the experiment, 
two similar cards marked with vertical stripes were present on certain 
trials. The two cards differed in that the stripes on one (always on the 
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left) were thin, and the stripes on the other (always on the right) were 
thick. On trials with these two cards a correct response was a jump to the 
left. On other trials, the stimuli were two plain gray cards which differed 
in brightness and the correct response was a jump to the right. One way 
of describing this problem is to say that the animals were rewarded for 
jumping to the thin lines and to the dark gray. 

Following training of this sort the groups were subdivided and given 
training with the same stimuli ; but, on half of the trials, each card was 
switched to the side it had not previously occupied. The various possible 
arrangements of the cards yield the four conditions indicated in Table 25 
as problems Ila, lib, and so on. For this problem, the conditions of rein- 
forcement were reversed for Groups I and II. Group I was required to 
jump to the left on trials with the striped cards and to the right on trials 
with the gray cards. The other group was required to jump to the rigid 
on trials with the striped cards and to the left on trials with the gray 
cards. In this part of the experiment the problem was exactly the same 
as it previously had been on half the trials, while on the other half the 
lateral arrangement of the stimuli was reversed. Now, if the specific pre- 
viously positive stimuli (thin lines and dark gray) were in complete 
control of the rats’ jumping response, this would suggest that the two 
groups would show a substantial, but equal, decrement on Problem II. 

Another way of looking at the experiment, however, leads to quite a 
different prediction. It might be argued that, in the original training, the 
rat merely learns to respond left on trials with the striped cards and right 
on trials with the gray cards, and that no discrimination between thick 
and thin stripes or between the two shades of gray need have occurred at 
all. On such an assumption, it will be observed that Problem II continues 
this arrangement for Group I and reverses it for Group II, Thus the 
shift to Problem II should produce little or no decrement for Group I, 
but a marked one for Group II. 

The results of the experiment appear in Figure 32.7. There was, it will 
be noted, a small and relatively temporary disruption of the behavior of 
the animals in Group I with the beginning of Problem II. This means that 
the exact stimulus conditions employed in Problem I had exercised some 
control over the rats’ behavior, and that changing these conditions pro- 
duced an interference through the mechanism of generalization decre- 
ment. Studies by Bitterman, Calvin, and Elam (3953) and Grico (1952) 
have independently suggested this conclusion. The effect on Group II, 
however, was much more marked, producing on immediate and persistent 
reduction in the number of correct responses. This seems to indicate that 
the chief thing the rats had learned in the first phase of the experiment 
was to jump one way in the presence of the striped stimuli and the other 
in the presence of the gray stimuli, and that the rever.nl of reinforced re- 
sponses had produced a substantial decrement through habit interference. 
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FIGURE 12.7. Results of experiment by Teas and Biuerman (1952). See text 

ior explanation. 
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stimuli and that the animal will continue to select the lighter. Moreover, 
this effect occurs even if the originally positive stimulus is paired with a 
still lighter one. That is, instead of choosing the originally reinforced 
stimulus, the animal now selects the new stimulus which bears the same 
relationship to the originally positive stimulus as the originally positive 
stimulus did to the negative stimulus. This transfer of discrimination to 
new pairs of stimuli is called transposition. 

The first experimental work on transposition was done by ICinnaman 
(1902) who demonstrated transposition in monkeys. Since then there 
have been scores of studies of transposition, which have shown that the 
phenomenon is real and that it occurs on a wide range of stimulus dimen- 
sions and in a variety of species. In the course of these studies, one other 
fact of importance has come to light. As the test stimuli are made more 
and more different from the training stimuli, there is a gradual weaken- 
ing of the transposition effect. This phenomenon lias been crucial in the 
development of a nonrelational explanation of transposition, nn explana- 
tion based upon the generalization of excitation and extinction. 

Building on this relationship between the degree of transposition and 
the difference between training and test stimuli, Spence (1936, 1937b) 
developed the theory suggested in Figure 32.8. This representation de- 
picts theoretical generalization gradients for excitation and inhibition 
along a size dimension. It is similar in conception to that presented in 
Figure 12.1 although the form of the generalization gradients is different. 

It should, perhaps, be emphasized that certain of Spence’s deductions are 
independent of the exact form of the generalization gradients. Spence, 
himself, has since suggested other shapes (Spence, 1942a). It also seems 
very probable that different species, different stimulus dimensions, dif- 
ferent discrimination measures, and, perhaps, different individual or- 
ganisms will require different generalization functions. 

The numbers on the baseline of Figure 12.8 have purely hypothetical 
meaning, although in Spence’s example, they were assumed to represent 
areas of visual stimuli. As is indicated in the caption to Figure 12.8 the 
gradients of excitation and inhibition are established by trnining with 
stimulus 256 as S-f and stimulus 160 as S— . Suppose now that a test for 
transposition is mode with the pair of stimuli 256 and 409. From the 
diagram it is evident that the strength of the tendency to react to 409 
(6.5) is greater than the tendency to react to 256 (4.8). Therefore, the 
organism will choose 409 in spite of the fact that 23G was the positive 
stimulus during training. For this pair of stimuli, transposition should 
occur. This is predicted, however, without any assumption that the or- 
ganism perceives the relationship “relatively larger.” Suppose, now, that 
the pair of stimuli 635 and 1049 are presented. The response strengths 
to these stimuli are 6.7 for 655 and 2.5 for 1019. Jn this ease, the reactmn 
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FIGURE 12.8. Spence’s theoretical treatment of discrimination. Areas of the 
visual stimuli used in the experiment are represented along the baseline. 
During the training, response to 256 is rewarded, response to 160 extinguished. 
As a result of reinforcing 256 the generalization represented by the solid lines 
results. That is, tendencies to react positively are generalized to the neigh- 
boring stim uli in proportion to the heights of the solid lines above each 
stimulus size. The non-reinforcement of 160 leads to a generalization of inhi- 
bition represented by the broken lines, extending also to stimuli on either side 
of 160. The resulting reaction tendencies, obtained by subtracting the negative 
from the positive, are indicated by the figures above each stimulus size. 
(Spence, 1937b). The use of the diagram to explain the results of trans- 
position experiments is described in the text. 


should be to the smaller of the two stimuli. That is, the opposite of trans- 
position should occur. 

The crucial test of Spence’s theory, as opposed to relational theory, 
obviously lies in the results of experiments designed to examine the de- 
tails of transposition behavior as a function of the difference between 
training and test stimuli. Early demonstrations of the fact that transposi- 
tion fails to occur under certain conditions are to be found in the studies 
of Gulliksen (1932), Kluver (1933), and Spence (1937a). This supplies 
indirect support for Spence’s theory. More recently several studies have 
demonstrated that the tendency for an animal to respond relationally in 
a transposition test decreases with the difference between test and train- 
ing stimuli and may show the predicted reversal (T. S. Kendler, 1950 ; 
Ehrenfreund, 1952). The strongest support for Spence’s theory comes 
from the experiment by Ehrenfreund. In this investigation, two groups 
of rats were trained in a T-maze to respond to the lighter (Experiment 
I) or darker (Experiment II) of two stimuli. For one group the positive 
stimulus was the darker of two bright stimuli. For the other the positive 
stimulus was the lighter of two dark intensities. After initial training, 
transposition tests were made with individual groups of animals. For 
those trained with the bright stimuli, the tests were with darker pairs; 
for those trained with dark stimuli, the tests were with brighter pairs. 
The results of the experiment appear in Figure 12 9 in terms" of the per- 
centage of relational responses in the transposition tests. It is quite clear 




TEST CONDITIONS 

FIGURE 12,9. Amount of transposition as a function of the difference between 
training and test stimuli. The test conditions. A, B, C, and D represent a series 
of approximately equal steps away from the training stimuli (Ehrenfreund, 
1952). 

that the results are in excellent agreement with predictions derivable 
from Spence’s theory: (1) There is a general tendency for the amount of 
transposition to decrease with increases in the difference between training 
and test stimuli. (2) There is also a point at which the opposite of trans- 
position occurs in that the curves fall below the 50 per cent point. This 
reversal was significant in the ease of Experiment J I, but not in the case 
of Experiment I. 

Results much less favorable to Spcnce r s theory have come from on ex- 
periment by Lawrence and DeRivern (1934). These investigators used 
the Lasbley jumping stand (Figure 30.13, p. 000) and a novel procedure 
to test for transposition. The stimuli were cards made up of two shades of 
gray, selected from a scries of seven grays ranging from number I (white) 
to number 7 (black). The bottom half of the training stimulus was always 
a mid-gray, number 4. The top half was lighter on half of the trials and 
darker on the other half. The method was a variation of the success live- 
presentation procedure. When the top half of the card was lighter, a 
correct response was a jump to the right ; when the top half was darker, 
the correct response was a jump to the left. Training consisted of practice 
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to a criterion on six different stimuli. Represented as a fraction with the 
numbers of the upper and lower grays on the stimulus cards as ^numerator 
and denominator, respectively, the stimuli were: 1/4, 2/4, d/4, b/% b/a, 
and 7 /4. To the first three of these, a right jump was correct ; to the sec- 
ond three, a left jump was correct. _ 

Tests for transposition were carried out over a long series of tnaLs with 
24 new cards which had a different shade of gray on the bottom. To illus- 
trate, one test stimulus, 3/1, was a card with a relatively light gray on 
the top half and white on the bottom half. Relational theory and absolute 
stimulus theory make different predictions regarding the response to this 
stimulus. Since both of the grays had previously been associated with a 
right jump, a theory of the type proposed by Spence would predict a 
right jump. Relational theory, on the other hand, predicts a left jump, 
since the upper part of the stimulus was darker than the lower and pre- 
serves the relationship previously requiring a left jump. On 12 pairs of 
stimuli in which the two thoeries make opposed predictions, 65 per cent 
were in favor of relational theory ; and this figure may be too low because 
the failures of the relational hypothesis occurred most conspicuously in 
cases where the upper and lower stimuli were fairly similar (for example, 
6/7). 

The two experiments described above present results which may be 
summarized as follows: Elirenfreund’s data are generally favorable to an 
absolute stimulus theory, supporting it in rather considerable detail. 
Lawrence and DeRivera, on the other hand, obtain results which support 
the relational interpretation, though with occasional failures for the pre- 
dictions from this theory to hold. One possibility is that some combination 
of the two theories will ultimately have to be developed (Stevenson and 
Bitterman, 1955). 

Another way of handling transposition in somewhat different terms has 
been suggested by a number of writers who propose to do so by means of 
a redefinition of the stimulus concept. If a stimulus is anything which can 
serve as a cue to which a response can be conditioned (Miller, 1959), it 
is an open question as to whether relationships can function in this way. 
If it turns out that they can, however, then the phenomenon of transposi- 
tion would be nothing more than a special case of stimulus generalization 
in which the stimulus dimension is one of similarity of relationships. In 
more mechanistic terms, this idea could be reduced to a matter of medi- 
ated generalization. Suppose that an organism learns to respond to the 
relationship, Si/S 2 , in which Si and S 2 are either different stimuli pre- 
sented to an animal simultaneously, or are two different parts of a single 
stimulus when the correct response depends upon the relationship be- 
y>.ee n t ese parts. If it is assumed that the act of comparing these two 
stimuli, r 0 provides proprioceptive stimuli which function as discrimina- 
tive stimuli, the following S-R analysis applies: 
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Si/S 2 > r 0 > *Si/S 2 * R 

This merely assumes that the response (R) is mediated by an internal 
stimulus ( s S x /S 2 ) which is somehow peculiar to the relationship involved. 
Obviously, to say that the organism “recognizes” or “perceives” the 
relationship is little different from making this assumption. Now, if the 
organism is confronted with a new pair of stimuli (S 3 and S<) bearing 
the same relationship to each other as Sj did to S 2 , it could be argued that 
the following holds : 


S 3 /S 4 > r p » *S 3 /S 4 

If *S 3 /Si is on some continuum with ‘Sj/S 2 but differs in absolute value, 
there should be generalization of the response (R) to the new relation- 
ship, and the degree of transposition should depend upon the similarity 
of training and test stimuli. There should, however, be no reversal of the 
transposition function with increasing differences between training and 
test pairs, as Spence’s theory requires. 

If the mediating response is one of observing or comparing, it might be, 
as suggested in connection with the simultaneous-successive discrimina- 
tion discussed earlier, that the former of these procedures would lead to 
better transposition than the latter. There is some evidence that this is 
the case. Baker and Lawrence (1951) trained rats to discriminate be- 
tween circles of two different sizes and then tested for transposition. They 
found that animals trained by the simultaneous method showed reliable 
evidence of transposition; those trained by the successive method did not. 
A possibility since suggested by Wertheimer (1959) is that the traces of 
relationships (si/s 2 ) are better retained than the traces of single stimuli, 
mailing transposition more likely following training on a simultaneous 
discrimination simply because the animal is more likely to remember the 
relevant cue. 

In much human behavior, implicit verbal labeling is a possible cue- 
producing response of the sort just described. On this basis it might be 
expected that responses to relationships would exhibit transposition to 
the extent that such responses occur. Kuenne (191C) and Alberts and 
Ehrenfreund (1951) have both reported studies which support this ex- 
pectation. Euenne ran four groups of children in a discrimination study, 
using positive and negative stimuli which differed in size. Then she tested 
for transposition with stimuli similar to the training stimuli (“near 
test”) and very different (“far test”). The children in the four groups 
had mental ages of 3, 4, 5, and 6 years. Their transposition scores were 
all very similar on the near test of transposition; but, as a function of 
mental age, they gave 50, 60, SO, and 100 percent relational responses on 
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to a criterion on six different stimuli. Represented as a fraction with tl 
numbers of the upper and lower grays on the stimulus cards as numerate 
and denominator, respectively, the stimuli were: 1/4, 2/4 3/4 5/4 ft/, 
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S i/S 2 » r 0 » «S,/S 2 > R 

This merely assumes that the response (R) is mediated by an internal 
stimulus (‘S 1 /S 2 ) which is somehow peculiar to the relationship involved. 
Obviously, to say that the organism “recognizes” or “perceives” the 
relationship is little different from making this assumption. Now, if the 
organism is confronted with a new pair of stimuli (S 3 and S 4 ) bearing 
the same relationship to each other as Si did to S 2 , it could he argued that 
the following holds: 


S 3 /S 4 » r p > *S 3 /S 4 

If *S 3 /S 4 is on some continuum with *Si/So but differs in absolute value, 
thm should be generalization of the response (R) to the nevr relation- 
ship, and the degree of transposition should depend upon the similarity 
of training and test stimuli. There should, however, be no reversal of the 
transposition function with increasing differences between training and 
test pairs, as Spence’s theory requires. 

If the mediating response is one of observing or comparing, it might he, 
as suggested in connection with the simultaneous-successive discrimina- 
tion discussed earlier, that the former of these procedures would lead to 
better transposition than the latter. There is some evidence that this is 
the case. Baker and Lawrence (1951) trained rats to discriminate be- 
tween circles of two different sizes and then tested for transposition. They 
found that animals trained by the simultaneous method showed reliable 
evidence of transposition; those trained by the successive method did not. 
A possibility since suggested by Wertheimer (1959) is that the traces of 
relationships (si/s 3 ) are better retained than the traces of single stimuli, 
making transposition more likely following training on a simultaneous 
discrimination simply because the animal is more likely to remember the 
relevant cue. 

In much human behavior, implicit verbal labeling is a possible cue- 
producing response of the sort just described. On this basis it might be 
expected that responses to relationships would exhibit transposition to 
the extent that such responses occur. Kpennc (1916) and Alberts and 
Ehrcnfrcund (1951) have both reported studies which support this ex- 
pectation. Ivuenne ran four groups of children in a discrimination study, 
using positive and negative stimuli which differed l'n size. Then she tested 
for transposition with stimuli similar to the training stimuli (“near 
test”) and very different (“far test”). The children in the four groups 
had mental ages of 3, 4, 5, and 6 years. Their transposition scores were 
all very similar on the near test of transposition; but, as a function of 
mental age, they gave 50, CO, SO, and 200 per cent relational responses on 
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tlie discrimination. (2) For the group originally trained to make the 
rough-smooth discrimination, the texture cue is still present, but irrele- 
vant. It should be more difficult for this group to learn the new discrim- 
ination. (3) The group originally trained with wide and narrow alleys 
should perform at a level between the other two. No positive or negative 
effect should transfer from the earlier experience, since the cues involved 
in the simultaneous situation are both new. The results of the experiment 
showed the predicted beneficial effect of the original training on the black- 
white discrimination, but failed to show the negative transfer expected 
in the case of the rough-smooth group. 

On the basis of this and subsequent evidence Lawrence proposed a two- 
process theory of discrimination learning, part of which is suggested by 
the schema in Figure 12.20. End-organ stimulation, according to this 
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view, initiates a mediating process which modifies the pattern of internal 
stimuli received by the organism. It is assumed that this modification 
occurs gradually with learning, making the internal pattern of stimuli 
more discriroinable after such experience. This acquired distinctiveness 
is thus available to aid later learning, even if no instrumental response is 
available to transfer to the new situation. 


THE FORMATION OF LEARNING SETS 

In many areas of learning, practice on a series of learning tasks lends 
to an improvement of the organism's ability to deal with learning situa- 
tions involved. This is a general statement which apparently applies at 
all levels of task complexity. Repeated conditioning and extinction in- 
crease the rapidity with which these processes occur (Bullock and Smith, 
1953). The number of trials required to learn a list of nonsense syllables 
decreases by about 50 per cent with practice on a series of n dozen lists 
(Meyer and Miles, 1953), And there is n remnrkablc improvement in the 
ability of organisms of several species to master discrimination problems 
with practice at making such discriminations. Harlow (1919, 1959) refers 
to such improvements as the formation of learning trts and explains learn- 
ing sets in terms of error factor theory, which will bo treated later. 

The classic study of learning sets (Harlow, 1919) was one in which 
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monkeys learned a series of over 300 discriminations in the Wisconsin 
General Test Apparatus (WGTA, Figure 12.11), each presented for 6 
trials. The changes in performance within these 6 trials for various blocks 
of discriminations are shown in Figure 12.12. What is striking is the 
great improvement in performance over the series of problems. On the 
final block the typical performance is chance (50 per cent on the first 
trial and nearly perfect from then on. Occasional errors do occur, for 
reasons to be discussed later. But these are relatively infrequent and ap- 
pear only once in about 20 trials, as is indicated by the leveling of the 
function for problems 201-312 at about 95 per cent. 

For the discrimination-learning situation, the development of learning 
sets is a case in which insightful behavior occurs as the end result of a 
great deal of experience with the sort of problem eventually handled in- 
sightfully. This may be true of insightful behavior in general. Harlow 
(1959) citing a variety of evidence (Birch, 1945; Schiller, 1952) comes 
to the conclusion that, at the present time, there is no good evidence for 
the existence of unlearned insightful behavior. 

Learning sets of this sort described by Harlow have been demonstrated 
in several primate genera (Miles and Meyer, 1956 ; Miles, 1957 ; Riopelle, 
1958), in rats (Koronakos and Arnold, 1957), in cats (Warren and Baron, 
1956), and in both normal and mentally defective children (Kaufman 
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FIGURE 12.11. 


The Wisconsin General test apparatus (Harlow, 1949 ). 
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DISCRIMINATION LEARNING CURVES 



TRIALS 

Prellminory 

Ditcrlmlnotlon* Discriminations 

*****•»■ 1-8 _. I-IOO 

— 9-16 101-200 

17.24 201-312 

— 25-32 

FIGURE 12.12. Performance on Trials 1-6 in a series of discriminations. The 
improvement with practice depicts the development of a learning set {Harlow, 
1949). 

and Peterson, 1958). 4 In genera), the speed with which learning sets de- 
velop, and perhaps the level to which they can develop, depend upon the 
phylogenetic level of the learner. Figure 32.13 presents data for the 
Rhesus monkey, squirrel monkey, and marmoset which support this con- 
clusion. The results obtained by Kaufman and Peterson indicate that the 
formation of learning sets is more rapid in either normal or mentally 
defective (IQ 50-75) children than it is in any of these species. K. S. 
Hayes, R. Thompson, and C. Hayes (1953) obtained data to suggest that 
learning sets in chimpanzees may develop at about the same rate as the 

‘The literature on learning acts Is quite large. The reader interested fn a more ex- 
tensive coverage of these materials would do well to begin with Harlow '* (19*?) 
paper. Some related papers, selected to cover a range of tasks, procedures and species 
nre: Harnett and Cantor (1957), Cotterman, Mover, and Wleken* (1956), Parby and 
Riopello (1955), Gatling (1951, 1952) and Hayes, Thompson, and Haves (1953a, 
1953b). Although they are not commonly treated In this connection. It seems possible 
that the studies demonstrating A transfer ot a discrimination atone some physical 
dimension will provide leads to additional factor* contributing to the formation of 
Icarnlngaet* (Baker and Osgood, 195tj I.awrenee, 1952). 
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FIGURE 12.13. Curves showing the formation of learning sets in the Rhesus 
monkey, squirrel monkey, and marmoset (Miles, 1957). 


mentally retarded children in Kaufman and Peterson’s study, and some- 
what less rapidly than in normal children. 

A special form of learning set occurs in discrimination-reversal experi- 
ments. In these studies the positive stimulus in one series of trials be- 
comes the negative stimulus in a subsequent series, and the previously 
negative stimulus becomes positive. Most of these studies have been done 
using rats as subjects, and either black-white or spatial discriminations. 
With a few exceptions (for example, Fritz, 1930) these studies have 
shown progressive improvement in the speed with which the discrimina- 
hon can be reversed (Krechevsky, 1932c ; North, 1950a, 1950b). At least 

non XAr” rat , tobe 1 M P aWe learning a ono-trial discrimina- 

(Burtendrik 1930 Vf SPatM disari ™‘™ situation 
( ytendijk, 1930 ; Dnfort, Guttman, and Kimble, 1954), And one study 

has shown that the process is remarkably similar in low-grade mental 

defectives (House and Zeaman, 1959). See Figure 12 14 * 
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REVERSALS 

FIGURE 12.14. Performance in a discrimination-reversal experiment com- 
pared for feebleminded human subjects and rals. In both experiments subjects 
first learned to respond to one position (for example, right) as correct. Then 
the problem was reversed, and the alternative position (left) became correct. 
When this was mastered the original response was again correct, and so on. 
Larger curve is for mental retardates. (House and Zeaman, 1959). 


Error Factor Theory and the Uniprocess Learning Concept 

To account for the learning-set phenomenon, Harlow has developed a 
theoretical treatment which he believes applies to all learning but is most 
appealing in the context of discrimination learning (Harlow, 1950, 1959). 
This theory is, first of all, a uniprocess theory; but it differs from other 
single-factor theories, such as those of Hull and Guthrie, in that the one 
mechanism is not, strictly speaking, learning at all. It is, rather, extinction 
or the suppression of incorrect responses. According to Harlow's view, 
the learner comes to the learning situation with the correct habit already 
available, but also with a number of other competing tendencies which 
obscure the correct response. The mastery of the discrimination by the 
organism consists in the elimination of these interfering tendencies. In 
his original analysis of discrimination learning, Harlow (1950) identified 
five of these interfering tendencies or error factors, four of which seemed 
most important to the understanding of the development of learning sets. 

1. Stimulus Perseveration. Stimulus-perseveration errors presumably 
reflect preferences which the organism has for the incorrect stimulus. 
These tendencies may generalize from previous learning (Jftrvik, 1953) 
or, like the preference for darkness commonly observed in rats, they may 
be innate. If they exist at nil, they appear ns n tendency for the animal to 
choose the negative stimulus on the first trial and to continue choosing it 
on subsequent trials. This source of errors is most important early in n 
series of discriminations. With increasing numbers of discriminations, 
the sire of the effect diminishes ns does the number of trials on which it 
can be defected (Harlow, 1959, p. 511; Riopelfe, 1953). 
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2. Differential Cue. In 1936, Spence pointed ont a second potential 
source of errors in many discrimination experiments. In a black-white 
discrimination, with black positive, the first reinforced response is not 
only to black, but also to (say) the left. Thus, there is an inherent am- 
biguity concerning the reinforcement contingency, and it is necessary for 
the response strength conditioned to the spatial cue to extinguish before 
a perfect brightness discrimination can occur. Operationally, this factor 
leads to a greater number of errors on trials in which the spatial position 
of the positive stimulus is changed than on trials where it is the same as 
on the previous correct trial. The influence of the differential-cue factor 
has been demonstrated in several studies and has also been shown to be a 
highly persistent source of errors (Braun, Patton, and Barnes, 1952) . 

3. Response Shift. It is a common observation in learning experiments 
of most sorts that absolutely perfect performance is difficult to obtain. 
The subject may go for a long series of trials without error and then, for 
no apparent reason, make a mistake. Harlow attributes this response-shift 
tendency to exploratory drive and reports that it seems incapable of 
complete suppression, at least in studies with the WGTA and food-incen- 
tives (Harlow, 1959, p. 517). 

4. Position Preferences. In discrimination experiments, responses may 
perseverate in a way that parallels stimulus perseveration described 
earlier. The source of such perseveration is often in “handedness,” which 
applies to lower animals as well as to man. In experiments where the 
positive and negative stimuli are shifted from side to side, such prefer- 
ences will lead to errors. The evidence is that position preferences are 
relatively unimportant as sources of errors in primate discrimination 
learning. They may determine the animal’s choice on the first trial in a 
new discrimination, but they are rapidly and easily suppressed (Riopelle, 
1953). In rats, on the other hand, such tendencies are sometimes very 
powerful. It is often reported, in descriptions of discrimination experi- 
ments, that position habits are so strong in certain subjects that the in- 
tended discrimination is never learned. 

The application of error factor theory to the development of learning 
sets involves the assumption that error tendencies can be suppressed fairly 
permanently, and that such suppression transfers at least to all new dis- 
criminations in the same general apparatus. If this is the case, the devel- 
opment of a learning set consists simply in the gradual elimination of 
error factors so that they do not have to be suppressed with each new 
discrimination. This explanation also appears to handle a certain puzzling 
fact about discrimination reversal. It has been shown (Reid, 1953) that 
overtraining on a black-white discrimination actually facilitates the learn- 
ing of the reversed white-black discrimination. In error-factor terms this 
means that the suppression of error tendencies may go on after a dis 
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crimination is learned. Since the elimination of such tendencies is as 
appropriate to the reversed discrimination as to the original one, it should 
assist the discrimination reversal. Put in these terms, it becomes apparent 
that error factor theoiy is really very similar to Kxeehevsky ’s theory of 
hypotheses. The idea which is basic in both formulations is that the re- 
sponses of the organism are determined at first by tendencies which are 
irrelevant to the solution of the problem. Obviously a perseverative tend- 
ency to respond by going to the left is operationally equal to a left-going 
hypothesis. Similarly, Harlow and Krechevsby both assume a sort of con- 
ceptual independence of error tendencies and the correct response, so that 
the elimination of one of these tendencies (hypotheses) is useful whatever 
the correct solution to the discrimination problem. The only important 
difference between the two positions seems to lie in the point that, accord- 
ing to Krechevsby, the animal does acquire a kind of knowledge in the 
situation, whereas Harlow does not regard the organism as learning any- 
thing, the entire “learning" process being attributed to extinction. Given 
Krechevsky’s adherence to the noncontinuity view of learning, it might 
be said that he has a one-trial learning theory and that Harlow has a 
no-trial learning theory. 

SUMMARY 

Although all learning involves a minor element of discrimination, spe- 
cial procedures are required to establish a fine differentiation between 
stimuli. In general it is necessary to maintain the strength of a response 
to one stimulus hy reinforcement and, at the same time, to extinguish by 
non-reinforcement responses generalized to an alternative stimulus. Two 
somewhat different techniques for accomplishing this objective arc to be 
found in the methods of successive and simultaneous presentation of 
stimuli. 

The method of successive presentation of stimuli is Pavlov’s method of 
contrasts and is especially well suited to the study of discrimination in 
the classical conditioning situation. The reinforced (positive) and non- 
rcinforeed (negative) stimuli are presented on randomly alternated trials, 
but never together. Under this procedure the animal gradually comes to 
respond on trials with the positive stimulus and not to respond on trials 
with the negative stimulus. 

In the method of simultaneous presentation of stimuli, the reinforced 
and non-reinforccd cues are presented nt the same time, and the situation 
is arranged so that the animal may respond to one of them, hy approach- 
ing it or manipulating it. If the subject responds to the stimulus arbi- 
trarily designated as correct (positive), it is rewarded. The most im- 
portant difference between this method of establishing a discrimination 
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and the method of successive presentation of stimuli is that the simul- 
taneous method affords a greater opportunity for the comparison of the 
positive and negative stimuli. This is a point to which we shall return. 


Conditioning-Extinction Theory 

Looked at operationally, discrimination learning procedures are a com- 
bination of two simpler procedures, those of conditioning and extinction. 
This fact has led several investigators to propose theories of discrimina- 
tion learning in which the processes of conditioning and extinction pro- 
vide the central concepts. These conditioning-extinction theories also take 
account of the fact that the relationship between positive and nega- 
tive stimuli necessitates a consideration of the process of stimulus 
generalization. 

Among the predictions to which conditioning-extinction theory leads 
are the following: (1) In discrimination learning, the level of responding 
to the reinforced stimulus should be less than would be obtained by the 
same number of reinforcements in an ordinary conditioning situation 
where no inhibition, generated by trials with the negative stimulus, would 
develop. (2) There will be a substantial level of responding to the non- 
reinforced stimulus whenever this stimulus is enough like the positive 
stimulus to receive, through stimulus generalization, response strength 
conditioned to the positive stimulus. (3) Increasing the ratio of rein- 
forced to non-reinforced trials, with the appropriate stimuli, should in- 
crease response level to both stimuli ; but there should be no improve- 
ment in the differentiation of the stimuli. (4) If an organism has acquired 
a response to a particular stimulus, the effect of beginning discrimination 
training should be to lower response strength to the positive stimulus. 
(5) The ease of developing a discrimination should vary directly with 
the psychophysical difference between the stimuli to be discriminated. 
Studies which vary widely in procedure have provided data in affirma- 
tion of all these deductions. 


Relational Theory 

In spite of the obvious power of the conditioning-extinction theory of 
discrimination learning, it has not gone uncriticized. As an alternative, 
many writers have favored a relational theory of discrimination. The 
central conception in relational theory is that the learning of a discrim- 
ination involves an active act of comparison, either between stimuli which 
are physically present, or between one stimulus which is present and the 
memory trace of another. One of the most direct deductions from such a 
theory appears to be that it would be easier to establish discriminations 
by the method of simultaneous presentation of stimuli than by the method 
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of successive presentation. Studies of the sort implied have sometimes 
yielded the predicted results, but with an embarrassing frequency the 
two methods have either produced no difference, or the successive method 
has actually been superior. There seems to be no simple or single explana- 
tion for this variety of outcomes. 

A second deduction from the relational theory of discrimination is that 
the transposition of a discrimination should be possible. Transposition is 
a variety of transfer of training in which a discrimination formed be- 
tween stimuli bearing a certain, objectively specifiable, relationship to 
each other will transfer appropriately to stimuli other than those involved 
in original training. To illustrate, an animal which has been trained to 
choose the larger of two particular circles will, if it exhibits the phe- 
nomenon of transposition, also choose the larger of two different circles 
Transposition has been demonstrated in many experimental situations 
It has been shown that an animal will actually reject a formerly positive 
stimulus in favor of a new one which bears the same relationship to this 
formerly positive stimulus as it did to the original negative stimulus. 
There is also evidence that the magnitude of the difference between the 
training pair of stimuli and test stimuli is important in determining 
whether transposition will occur. This last fact has provided the basis for 
the development of an S-R theory of transposition. This theory explains 
transposition in terms of interacting gradients of generalized excitation 
and inhibition. Experimental evidence specifically relevant to the ques- 
tion “Which of these theories is correct!” is mixed. In order to obtain 
an adequate explanation of transposition, it will ultimately be necessary 
to develop a different theory, perhaps one which begins with some as- 
sumption capable of making a relationship a cue. Another set of extremely 
important considerations stems from the fact that the initial discrim- 
inability of stimuli is partly determined by previous experience with 
these stimuli. 

Learning Sets 

A final set of factors which increase the difficulties involved in under- 
standing discrimination learning derives from the fact that the ability to 
deal with discrimination problems improves with practice. Animals de- 
velop learning sets. In order to account for the formation of learning 
sets, Harlow has proposed a theory of discrimination learning in general, 
in which the correct response is assumed to be present at the beginning of 
training in a particular discrimination situation, hut suppressed by error 
tendencies. The nppearance of the correct response is accounted for in 
terms of the extinction of these erroneous responses. Among the important 
error tendencies to he considered are preferences for particular stimuli 
or responses, tendencies to alternate, and confusion stemming from the 
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fact that, in most discrimination-learning experiments, the aspect of the 
correct response which determines its correctness is ambiguous. The de- 
velopment of learning sets theoretically depends upon the extinction of 
these error tendencies and the transfer of this extinction to new 
discriminations. 



13 


Motivation and Learning 

The topic op motivation is an increasingly important field of investiga- 
tion in its own right and has been the subject of several book-length 
expositions (for example. Young, 1936; Bindra, 1959). Discussions of 
motivation differ widely in their overall treatment of the subject, and 
also in the relationship which motivation is assumed to have to learning. 
Because of this diversity, it may be well to begin what we have to say 
with a brief statement of the general position to be adopted. 

METHODOLOGICAL STATUS OF THE CONCEPT 
OF MOTIVE 

It is important to recognize at the out set that motivation is a concept 
(Koch, 1941a, 1941b), a concept suggested by certain characteristics of 
behavior which are a matter both of everyday observation and of experi- 
mental manipulation in the laboratory. The two features of behavior 
which seem to demand the development of some such concept ns that of 
motivation are (1) certain variations in the behavior of the same indi- 
vidual from time to time, and (2) certain extreme individual differences 
in response to the same situation, Including differences in the apparent 
speed of learning. If, for example, we observe that a 1 0-y car-old youngster 
loiters and dallies along his way to school and, a few hours later, rushes 
pell me 11 on his way to the playground, we are apt to infer that his 
motives differ on the two occasions. Or if we watch two small children in 
the presence of a large dog and see that one of them runs in apparent 
terror to its mother, while the other shows strong interest or at least no 
alarm, it again seems reasonable to think In terms of different motives 
and to say that the first child was afraid of the dog. 

In order to make cither of these inferences with certainty, it is essential, 
first, to be able to exclude an alternative explanation in terms of rapacity, 
skill, or learning. In the first example, it is necessary to lie certain that 
the boy has not learned to run faster in the interval separating the two 
observations; in the second example, we must he willing to assume that 
the two children do not differ in their innate responsheness to dogs This 
3?' 
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requirement implies a distinction between motivation on the one hand, 
and the capacities of the organism on the other. In theories such as IIu 1 s, 
where this distinction is sharply made, the assumption is that motives are 
relatively temporary conditions of the organism which serve to energize 
relatively permanent habits or capacities. 

Drive and Drive Stimulus 

The motivation of behavior comes about through the existence of con- 
ditions (drive-establishing operations) which release energy originating 
in the organism’s metabolic processes. This energy, in and of itself, is 
directionless and may serve any of a variety of motivational objectives. 
As Hebb puts it, . . drive is an energizer, but not a guide ; an engine, 
but not a steering gear.” (Hebb, 1955, p. 249). Against this view of 
motivation as directionless there stands a considerable body of evidence 
that motivated behavior is directed. The hungry rat, given all three op- 
portunities, eats rather than drinks or copulates, suggesting that motives 
somehow provide for the direction of activity. The mechanism commonly 
accepted as subserving this function of guiding motivated behavior is 
that of drive stimulation (S D ). There is good reason to believe that most 
drives have associated drive stimuli. Some of the evidence will be pre- 
sented later in this chapter. For the present, there are several methodo- 
logical observations which need to be made. 

The basic question to be answered is whether drive and drive stimuli 
are two separate processes or two reflections of a single one. Our view is 
that they are two processes and that it is important to keep them separate. 
It has sometimes been urged that only one process is involved, and that 
the concept of drive stimulus is superfluous. Because the conditions for 
producing drive stimuli are the same as those for creating a drive, the 
argument is that this fact makes drive and drive stimulus operationally 
indistinguishable. This point does not seem well taken because, (a) even 
admitting the identity of establishing operations, it is entirely possible 
for drive and drive stimulus to differ with respect to other characteristics. 
Presumably, drive stimuli would share the properties of other stimuli 
and would function in such a way as to make the discrimination among 
drives and drive-stimulus generalization possible. Drive, considered solely 
as an energizer would not serve such functions. (6) There is also the 
strong likelihood that the functions which relate drive and drive stimuli 
to these identical determining conditions will be different. In the case of 
hunger, for example, there is evidence that stomach contractions, which 
represent the source of drive stimulation (Cannon, 1934), reach their 
maximal strengths within a few hours, although the energizing effect of 
drive continues to increase for considerably longer, (c) Furthermore if 
one of these two concepts is superfluous, it is at least as likely to be drive, 
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itself, as it is to be drive stimulus. Estes (1958) makes a good case for 
this point, (d) Finally it is important to point out that recognizing the 
existence of drive stimuli does not require the postulation of a steering or 
guiding function of drive per se. The two processes, as Hebb cogently 
argues, may profitably be considered separately, with the drive or motive 
concept being reserved for the energizing function. 


Drive as an Intervening Variable 
As we mentioned earlier, drive is a concept, or an intervening variable. 
It is an unobservable construct employed as a logical link between ma- 
nipulated antecedent drive-establishing operations and the effect of these 
operations upon behavior. In this respect the concept, drive, is like that 
of learning. We assume that it is produced by certain environmental 
circumstances and reflected in certain effects upon behavior. Figure 13.1 
is a diagrammatic representation of this general point. It depicts the 
logical status of drive as seen by different theorists, and it also shows how 
they think of the concept of drive stimulus as fitting into this scheme. 


HULL (1943) 



MILLER AND DOLLARD (1941) 
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As Figure 13.1 shows, drive and drive stimulation are intervening vari- 
ables in all of these conceptions. There is, however, little agreement as to 
the ‘ ‘ true nature’ ’ of drive, and even less on the nature of the relationship 
between drive and drive stimuli. Hull’s conception was that drive-estab- 
lishing operations are such as to create a state of need in the organism, 
and that drive and drive stimulation both derive from it (Hull, 1943, 
p. 72). Miller and Dollard propose a stimulus theory of drive according 
to which motives result from high intensities of drive stimulation. Estes 
goes a step farther and develops a theory in which the only considerations 
are the stimuli produced by drive-establishing operations and the re- 
sponses conditioned to them. In this theory no special concept of drive is 
necessary. 


S-R and R-R Concepts of Motive 

In various places, Spence (for example, 1944, 1948) has distinguished 
between two forms of psychological law which he refers to as S-R and 
R-R laws. S-R laws have terms on the independent-variable side, which 
refer to manipulations of the environment (loosely, “stimuli”). The R-R 
laws, in contrast, are formulated between independently observable as- 
pects of a single organism’s behavior. Corresponding to the difference 
between S-R and R-R laws, it is possible to distinguish two general classes 
of operations for introducing concepts into the language of psychology. 
On the one hand, there are concepts which are defined in terms of ante- 
cedent conditions, in Spence’s treatment, “stimuli.” On the other hand, 
there are other concepts which are defined in terms of the organism’s 
behavior or of its physiological state. The essential difference seems to be 
that the R-type concepts require a reference to characteristics of the or- 
ganism. The S-type concepts do not. 

It often happens that definitions of both types are available for terms 
bearing the same construct name. This is true of the concept, “motive.” 
There are times, particularly in the case of complex human motives, when 
it is important to remember the operational dictum that different opera- 
tions define different concepts, and that the equivalence of differently 
defined concepts requires demonstration. Whether anxiety produced by 
a traumatic experience (an example of an S-defined concept) is the same 
as anxiety reflected in a clinical interview (an example of an R-defined 
concept) is a question of fact, which the terminological identity of the 
concepts does nothing to resolve. Table 26 is an attempt to clarify the 
nature of the concept of motive still further, by including additional 
details and by dealing with a selection of specific motives. The S-R and 
R-R types of concept are treated separately, and an attempt has been 
made to illustrate the main independent variables involved in the defini- 
tion of each. An effort has also been made to categorize the important 
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manifestations of motives on the response side. The general categories 
which seem to be important in this last connection have been indicated in 
detail only for hunger. It seems likely, however, that the same categories 
apply to other forms of motivated behavior. 

TABLE 26 

Summary or Some Different Conceptions of Motivation 


A. S-E Conceptions 
Intervening Variable 

Independent Variable (5) (dfoftce Defined ) Dependent Variable (E) 


1. Time (for example, food 
deprivation) 

2. Stimulus intensity 
(for example, shock) 

3. History of organism 
(for example, avoidance 
conditioning) 

Hunger 

Pain Avoidance 

Anxiety 

Consummatory Response 
(eating) 

Instrumental Response 
(Performance in 
maze situation) 
Substitute Response 
(Thinking about food) 

Escape 

Avoidant Behavior 

Independent Variable (E t ) 

B. ll-li Conceptions 

Intervening Variable 
Motive Defined 

Dependent Tort'aMe (E,) 

1. Response strength 

Hunger 

Hare performance. 

(for example, eating) 

2. Score on test 

Anxiety 

Performance on learning 

(tor example, anxiety 
scale) 

3. Physiological condition 

Sexuality 

task. 

Performance for n sexual 

(for example, androgen 
level) 


lure. 


The most important points suggested by Table 2G are the following: 
(1) ‘Whether a particular item of observation appears as an independent 
or dependent variable in the diagram depends upon the use to which it 
is put. Consummatory behavior may function in cither way, as the two 
examples employing the motive, hunger, show. (2) It seems quite likely, 
in the case of the S-tvpc specification, that all three of the major classes 
of independent variables are involved in determining the strengths of 
these motives. The evidence on this point is clearest for the appetitise 
drives. The strength of a hunger-motivated response depends upon de- 
privation, the desirability of the particular food, and the organism V 
previous experience with it (Ghent, 1951, 1957). Similarly the strength 
of sexual behavior depends uj>on deprivation, upon the attractiveness of 
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the sexual lure, and upon sexual experience. (3) The categorization of 
motivational expressions into consummately, instrumental, and substitute 
modes is offered only very tentatively, but there is evidence from a wide 
range of motives that something like such a classification may be useful. 
The consummatory and instrumental categories probably require no 
further comment, but the generality of substitute or indirect expressions 
of motives may not be so obvious. At the human level, the fact that un- 
satisfied drives are expressed in substitute, indirect, and imagined ways 
is well established. According to the ethologists, such substitute behavior 
also occurs in lower animals. For example, a wild bird raised in captivity 
may display a complicated pattern of behavior appropriate for catching 
insects, even in its cage where there are no insects (Lorenz, 1952). Simi- 
larly, male rats have been observed to make sexual advances toward other 
males, or even toward inanimate objects when no female is available. 

The Importance of the Response Measure 

The existence of this variety of motivational expressions of course 
raises the question whether all of them reflect motive strength in the same 
way. The evidence points very clearly to the conclusion that they do not. 
Although most measures of the strength of hunger increase with depriva- 
tion, many show a decrease with long periods of deprivation (Skinner, 
1938, p. 396), and some show little change or even a decrease from the 
beginning (Campbell and Sheffield, 1953; Strong, 1957). Analysis of the 
factors which produce these differences indicates that the main effect of 
hunger upon the responses of the rat is to increase the amount of loco- 
motor activity displayed. Smaller responses, such as grooming and biting 
at the cage, are much less affected. Thus, indices which reflect the level of 
gross actions of the whole body tend to increase with deprivation ; those 
reflecting fine, nonlocomotor, responses do not (Strong, 1957). 

Even among measures which increase with deprivation, there are dif- 
ferences in their sensitivity and in their ability to reflect differences at 
all levels of drive. Common sense would probably say that consummatory 
behavior would be the most accurate indicator and that other responses 
would reflect drive level less faithfully. In this case, common sense turns 
out to be misleading, as has been shown very effectively in a series of 
studies by N. E. Miller and his associates (N. E. Miller, Bailey and 
Stevenson, 1950; N. E. Miller, 1955, 1956, 1957). The first of these eiperi- 
ments was an investigation of the behavior of rats made hyperphagic 
by a brain lesion. It had previously been discovered (Hetherington and 
Hanson, 1942) that animals with lesions in the region of the ventromedial 
nuclei of the hypothalamus overeat enormously. The evidence leads one 
to conclude that the hypothalamic lesions must have had a pronounced 
effect upon the hunger of these animals, and that their motivation would 
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be higher than that of normal subjects. To test this hypothesis, Miller, 
Bailey, and Stevenson compared operated animals with normal controls 
on a number of tasks and found exactly the opposite to be true. On any 
task which required the animal to overcome an obstacle to get food, the 
hypothalamic subjects were much inferior to normals. The tests included 
rate of bar pressing, the force with which the animals would pull toward 
food in a straight alley, the amount they would eat when they had to 
lift a heavily weighted lid of a food receptacle to obtain food, and the 
amount of quinine (which made the food bitter) jn their food which they 
would tolerate. 

The fact that all of these measures revealed decreased drive in the 
hyperphagie animals, whereas food intake had suggested increased drive, 
seems to mean that food intake depends upon different factors from those 
controlling performance on the other measures. The results of other 
studies support this interpretation. Up to about 24 hours of deprivation, 
various measures of the hunger drive agree quite well (Ilorenstcin, 1931 ). 
Beyond this point, marked discrepancies occur. The bar-pressing and 
quinine measures previously mentioned continue to increase for as much 
as 4 days. Food intake, on the other hand, seldom increases beyond 21 
hours and may even show a decrease, which is often taken as showing a 
weakening (inanition) component resulting from starvation. 

A set of results obtained on rats (Miller, 1956), illustrating the changes 
in various measures with deprivation, appears in Figure 13.2. One of the 
measures included in this presentation is an index of the activity of the 
stomach. To obtain this measure an inflated balloon on the end of a plastic 
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function of number of hour* of food «lrprlr*tion (S, E. Miller, 19^6). 
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point of this fact and, extrapolating the function to zero, concluded that, 
under conditions of satiation, something like 25 per cent of maximal 
response strength remained. Later studies (Koch and Daniel, 1945; 
Saltzmann and Koch, 1948) have shown this conclusion to be in error. 
With less than two or three hoars of deprivation the empirical function 
shifts sharply downward, and response strength at very low levels of 
motivation is considerably weaker than Hull's extrapolation had led him 
to suppose it would be. 

In a study which sampled a number of deprivation points and also 
investigated the effects of prefeeding upon performance, Kimble (1951) 
trained rats to push open a small swinging door to obtain food and used 
the speed of this response as a measure of the effect of drive on perform- 
ance. Tests were carried out under a variety of motivational conditions, 
some of them produced by allowing the rats to cat a portion of their daily 
meal just before the panel-pushing test. In other cases the tests were 
administered after some amount of food deprivation up to 24 hours, 
which was the training deprivation. The results of the experiment appear 
in Figure 13.3 in terms of response speed. Clearly, prefeeding progres- 



TIME IN HOURS 

rfCLTRE 13.3. Changes with response strength b» a function of the fnjresfion 
of food (left-hand portion of the graph) and food drpritatjon (Kimble, 
1951). 

sivcly slows np the speed of responding, and deprivation leads to faster 
reactions. The reason for the various discontinuities which appear in 
Figure 13.3 are not clear. It may be that the initial horizontal, or perhaps 
slightly downward, trend lasts ns long as food remain* in the stomach and 
distends it. The time values are about right. The Midden epurt at 2-3 hour* 
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may be a residual reflection of the fact that rats allowed ad libitum access 
to food put themselves on a roughly rhythmical, 2-4 hour, feeding sched- 
ule. This schedule may, in turn, result from the fact that stomach con- 
tractions and hunger pangs become intense at about this interval. 

Considerations of this sort led Eisman (1956) to propose a theory in 
which the hunger drive is hypothesized to depend upon two parameters, 
one controlled by the maintenance schedule of the animal, the other by 
deprivation in the usual sense. The second factor is regarded specifically 
as depending upon hunger contractions and reaching an asymptote early 
in the deprivation period. Eisman’s guess was that the upper limit oc- 
curred in 4 hours or so. Miller’s evidence (Figure 13.2) indicates that the 
limit of this function may occur slightly later. In a general way, the pre- 
dictions to which Eisman’s theory leads is that increasing the depriva- 
tion period within some fixed interval will increase motivation and, there- 
fore, response strength. This results from the influence of the first hunger 
parameter. With values of this parameter constant, there should be no 
effect of deprivation beyond approximately the first four hours (Eisman, 
Asimow, and Maltzman, 1956) . 

To investigate the validity of this explanation, Eisman (1956) ran 5 
groups of rats in a black-white discrimination problem. Three of the 
groups were starved for 47 out of every 48 hours and run 4, 22, or 45 
hours after feeding. The prediction was that these three groups would 
perform equally well, since the first parameter was the same for all groups 
and the value of the second was, in every case, asymptotic. One of the 
other two groups was starved 24 hours out of every 48 and run 22 hours 
after feeding. The last group was starved 5 hours out of every 48 and 
run 4 hours after feeding. These last groups were, thus, run under condi- 
tions when the second parameter was asymptotic, but the first would 
have a decreased value. These animals should perform less well than any 
of the animals on the 47 -hour maintenance schedule. There was no differ- 
ence in performance among the animals in the first three groups. The 
animals starved 24 hours out of 48 were somewhat inferior, and the 
animals starved only 5 hours out of 48 were poorest of all. 

These results support Eisman’s theory for the particular learning 
situation he employed and show the importance of the feeding schedule 
in determining level of drive. They do not, however, agree at all with 
the results obtained by other investigators, some of whom have obtained 
differences in drive level where Eisman’s hypothesis would indicate that 
there should be none. Yamaguchi, in the study described above employed 
such conditions. Specifically, all of his animals were starved for 72 hours 
out of 120 and were tested at the appropriate time within this interval 
Since this procedure equated the subjects on the first parameter there 
should have been no differences among groups tested at deprivation 
intervals greater than four hours. There were such differences, however, 
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a fact which seems to refute Eisman ’s theory about the second drive 
parameter. The importance of the first (maintenance schedule), however, 
remains and deserves more study than it has received. 

The Role of Competing Responses 
The most likely explanation of the differences between Eisman ’s re- 
sults and those of others is in the response measure employed. We have 
had a number of occasions to call attention to the fact that motivational 
variables seem to have more influence on speed than on measures of 
accuracy in learning. This, in turn, may come about as a result of the 
fact that the measures which vary most with drive arc particularly 
susceptible to disruption by the effects of competing responses. The sug- 
gestion is that at least a part of the increase in speed of responding, and 
in measures which involve speed, is to be attributed to the fact that less 
time is spent in extraneous activity. Cotton (1953) studied the runway 
behavior of rats under different conditions of hunger. After 44 initial 
training trials, the same animals were observed under 0, 6, 16, or 22 
hours of food deprivation and run under each drive condition until they 
made at least 49 responses without competing responses. A competing 
response was defined as any response which interfered with progress 
toward the goal. Stopping to wash, sniffing at the side of the alley, and 
other exploratory activity are typical examples. The procedure made it 
possible to obtain functions of hunger from the trials on which competing 
responses did nnd did not occur. The running time measures obtained 
Under these two conditions are presented in Figure 13.4. These results 
are instructive in a number of ways. (1) It is obvious, in the first place, 
that the animals running under low drive indulge in much more extrane- 
ous behavior than animals running under high drive. (2) The fact that 
the two curves nearly come together at the highest drive level means that 
competing responses have been almost completely eliminated at this drive 
level. (3) Although the drive function is considerably reduced in magni- 
tude when competing responses arc eliminated, the slight slope appearing 
in the lower function of Figure 13.4 is significant. (4) Together these 
facts mean that increasing drive lias two effects upon the running be- 
havior of the rat in the straight maze. The first of these is a strengthen- 
ing of the running response. The second is the elimination of competing 
responses. Recent research suggests that the first of these effects is more 
important than Cotton’s results indicate. R. A. King (IP59) performed 
an experiment which replicated the Cotton study in most respects, except 
that he used independent groups of rats at each drive Ie\rl. Kings 
results are shown in Figure 13.5. Again the elimination of competing 
responses decreases the slope of the drive function, but in this case much 
less than in Cotton’s study. In King’s experiment, in fact, the main 
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FIGURE 13.4. Response strength as a function of number of hours of food 
deprivation measured on trials with (upper curve) and without (lower curve) 
competing responses (Cotton, 1953). 

influence of drive increase appears to be upon the running response itself. 

The recognition of the fact that drive level may control the qualitative 
characteristics of behavior has a number of important consequences, one 
of which is that it introduces a completely new dimension into the analy- 
sis of the effect of drive upon behavior. This dimension, in the absence 
of a commonly accepted name, is one which we may call response compati- 
bility, by which we mean the degree to which the responses occurring 
under a given level of drive are consistent with those required by the 
learning or performance situation. To illustrate the importance of this 
variable, we borrow an example from the Gestalt psychologists, the 
umweg or detour problem. 

Consider the situation represented in Figure 13.6 in which an animal, 
in the classical experiments a chicken, is placed before a fence on the 
other side of which there is a dish of grain. In order to get to the grain 
it is necessary to turn directly away from it and take the detour around 
the enclosure. The chicken, being of somewhat limited intellectual capaci- 
ties, finds this solution difficult. Instead of trying to find ways around 
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FIGURE 13, 5. Response strength as a function of food deprivation measured 
with competing responses included (Class I responses) and excluded (Class 
II responses). Adapted from R. A. King (1959). 

the barrier, it tends to keep on making futile efforts to get through the 
fence and directly to the goal. Learning occurs, but only slowly. 

Now consider the effect of increasing hunger upon the chicken’s be- 
havior. As in the case of Cotton’s rats, the chicken tends to restrict its 
actions more and more to those which seem to lead directly to food. But, 
where this effect improves performance in the straight runway, it does 
exactly the opposite in the detour situation. Increasing motivation inter- 
feres with performance because the behavior associated with increased 
motivation is incompatible with that required by the problem. 

The concept of response compatibility has an obvious implication for 
the function relating drive to performance, namely, that there is no 



FICURE 13.6. The umrreg (detour) problem. The organism at point S tne*t 
taV.e the path indicated if the dallied line to food which fi vfiibtc Inrooffi 
the icrceu. 



408 CONDITIONING AND LEARNING 

single function which can be expected to apply to learning situations in 
general. The performance curve may be constantly increasing, constantly 
decreasing, or curvilinear, depending upon relationships of the sort con- 
sidered above. At the same time, it is important to recognize that, behind 
this statement, there lies the assumption that the function of dri\e on 
behavior is "basically an increasing one. The responses given increased 
energy may, however, be either appropriate or inappropriate as far as 
the demands of a particular situation are concerned. It is this latter 
characteristic which makes possible what superficially appears to be a 
multiplicity of functions. 

In a more formal way the sense of this interpretation may be expressed 
in terms of Hull’s (1943) theory. Hull proposed that drive (D) multi- 
plies with (energizes) habit strength (H) to produce excitatory poten- 
tial (E), and that response strength (R) depends upon the value of E. 
In summary : 

R = f(E) = f(D) Xf(H). 

This statement, however, applies only to single-response situations. "Where 
more than one response is possible, an equation of the same sort is neces- 
sary for each. That is : 

R 1 = f(E 1 ) =f(D) X f(H x ), and 
R 2 = £(E 2 ) = £(D) X f (H 2 ). 

When more than one response is possible, the one which will probably 
occur in the choice situation depends upon which E is stronger, and 
since D multiplies both habits, this will depend upon which habit is 
stronger. For the same reason the difference between E x and E 2 depends 
upon the difference between H x and H 2 . Specifically: 

Ei — E 2 = f(D) X f (Hi - H 2 ). 

With this last equation formulated it is possible to make two important 
points: (1) The difference between E x and E 2 will increase with D, thus 
increasing the probability of occurrence of the response depending' upon 
the stronger habit. (2) Whether this is beneficial or hindering so°far as 
learning is concerned depends upon whether the stronger response is 
correct or incorrect. Such theorizing is important in much of the current 
work on human motivation, some of which will be discussed in the next 
chapter. 


Behavior in the Absence of the Acquisition Drive 

There is evidence from a considerable variety of experiments that a 
kabU acquired under a certain drive will sometimes appear when that 
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drive is satiated (Teel and Webb, 1951). Naturally this raises the ques- 
tion of the conditions responsible for this phenomenon. Webb (1952) 
has surveyed the literature in this field and has suggested a three-fold 
classification of the explanations for responses which occur in the absence 
of the acquisition motive. Certain of these explanations are merely 
nominal and only provide another name for the phenomenon. Wood- 
worth’s (1918) proposal that well-learned hnbits become ends in them- 
selves and G. W. Allport's (1937) concept of functional autonomy are 
examples. A second set of explanations are referred to ns acquired- 
tendency theories. In these explanations it is assumed either that training 
serves to condition the original drive state to stimuli in the learning 
situation, so that a motive is actually present when the training drive is 
satiated, or that drive is not necessary for the elicitation of a response 
once it is learned. Finally there is irrelevant-drive theory which assumes 
that drives other than those under which a given habit was acquired are 
capable of giving it energy. 

Theorists as diverse in their views as Freud and Hull have held the 
irrelevant-drive position. Freud, in particular, made much of the point 
that libidinal energy was capable of expression in substitute form and 
could be made to serve nonsexual purposes. Hull postulated that all 
drives, relevant (D) and irrelevant (D), added to produce a generalircd 
drive state (D). The further assumption that reaction potential ( b Er) 
was a multiplicative function of habit strength (sITn) and generalized 
drive (B) leads to the prediction that irrelevant drives can energize 
behavior. 

Experimental evidence on this particular point is mixed, noth positive 
and negative results have been reported. On the positive side, Spence 
and his associates have had considerable success in demonstrating that 
manifest anxiety, measured by a paper and pencil test, functions ns an 
irrelevant motive in learning situations ranging from eyelid conditioning 
(Taylor, 1951) to verbal learning (Montague, 1953) and stylus maze 
learning (Farhcr and Spence, 1953). Amsel (1950) found that rnts per- 
formed a running habit motivated by fear better when they were hungry 
than when they were satiated. On the negative side, Franks (1957) failed 
to find an increase in the rate of eyelid conditioning when drive was 
increased by deprivation from food, drink, and tobacco. And it has bom 
found that injections of cocaine produce no increase in the resistance to 
extinction of a bar-pressing response over nnd above that which can be 
attributed to heightened general activity (Miles, 195S). 

A series of studies begun by Webb (1919) seemed at first to show that 
responses acquired under food motivation could be energized by thirst. 
Webb trained hungry rats to push open a small swinging door to grt 
food nnd then tested them in extinction trials under different degrees of 
water deprivation. He found that increasing thirst led to inerr.ving 
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resistance to extinction, a finding later confirmed by Brandaner (1953). 
These studies, however, have been severely criticized on the ground t lat 
thirsty rats do not eat as much as nonthirsty ones (Yerplanck and 
Hayes, 1953) and therefore must have been somewhat hungry in the 
extinction tests in the Webb and Brandauer investigations. To test this 
possibility Grice and J. D. Davis (1957) repeated the Webb experiment 
using four different groups of rats. AH of the animals learned the panel- 
pushing response hungry, but they were extinguished under four differ- 
ent conditions. During extinction one group was hungry; one was sati- 
ated for food and water ; one was thirsty but allowed free access to food ; 
and one was deprived of water with food available until just before the 
extinction test and then it was allowed to drink. The important group is 
this last one. If thirst inhibits eating and allows the animal to get 
hungry, the opportunity to drink just before extinction might be ex- 
pected to eliminate the inhibition, causing the rat to respond in terms 
of its hunger. This is what seems to have happened in this experiment. 
Table 27 presents median numbers of responses to extinction, speeds of 

TABLE 27 


Results During Extinction Tests 
(F rom Grice and Paris, 1957) 


Group 

Median Number 
of Responses 

Median 

Speed 

Median Responses 
Per Minute 

Hungry 

26.5 

246 

4.68 

Satiated 

13.5 

105 

1.29 

Thirsty 

13.0 

71 

1.18 

Thirsty-drink 

22.0 

218 

3.62 


the first 10 responses, and overall response rates. In each case it is quite 
clear that the last (thirsty-drink) group responded much more vigor- 
ously than the thirsty group. Thus for this particular situation, the indi- 
cation is that the irrelevant thirst drive does not actually energize the 
response learned on the basis of hunger. Instead the suggestion is that 
thirst, in the earlier studies, had merely prevented satiation of hunger, 
winch was responsible for the apparent effect of the irrelevant motive'. 

Although the Grice and Davis demonstration casts considerable doubt 
upon the validity of the particular idea that thirst is capable of energizing 
habits originally learned under a hunger drive, it should not be con- 
cluded that this negates the entire concept of irrelevant drive. The posi- 
tive evidence cited earlier still stands. Obviously much remains to be 
learned about the conditions under which such motives may energize 
behavior or fail to do so. 
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THE EFFECT OF DRIVE ON LEARNING 

A problem of great theoretical interest concerns the relationship be- 
tween motivation and learning. The basic question is the following: Is 
drive one of the variables determining the strength of habit, or is it 
exclusively a performance variable? The experimental procedure for 
determining the answer to this question is essentially the two-phase 
method previously discussed in connection with conditioned stimulus 
intensity (p. 119). Different groups of subjects are trained initially 
under two or more different levels of motivation. Then, at some point, 
each of these groups is subdivided. Some of the subjects continue under 
the original drive condition; others switch to the alternative drive levels. 
For the simplest situation, where only two drive levels are involved, this 
procedure generates the 2x2 factorial design represented in Table 28, 


TABLE 28 

Experimental Desicn for Separating Effects of Drive on 
Performance from Effects of Drive on Learning 




Later Test Drive 



High 

High 

Measures obtained 

Low 

Measures obtained 

Maw gC 

Initial 

Training 

Drive 

Low 

in test condition 

Measures obtained 
in test condition 

in test condition 

Measures obtained 
in test condition 

li 

M u * || 



M B |ft Mu* 

Means reflecting performance differences 



in which the rows represent the conditions of original training ami the 
columns represent conditions after drive levels have been switched for 
some of the subjects. As with the CS-infcnsity example, differences in 
the column means represent performance differences, and differences in 
the row means reflect learning differences; that is, differences due to the 
original training drives arc now reflected with drives equated. 

There are six points which should be emphasized with respect to this 
diagram: (1) It is to be noted that the measures in the table are lhn%e> 
obtained in the Bccornl phase of the experiment. (2) This second phase 
lias taken a number of different forms in different studies. The test has 
sometimes been in extinction, sometimes in recall, ami sometimes in a 
series of further reinforced trials. (3) The measures of behavior em- 
ployed have includes! resistance to extinction, running time and its 
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THE EFFECT OF DRIVE ON LEARNING 

A problem of great theoretical interest concerns the relationship be- 
tween motivation and learning. The basic question is the following: Is 
drive one of the variables determining the strength of habit, or is it 
exclusively a performance variable? The experimental procedure for 
determining the answer to this question is essentially the two-phase 
method previously discussed in connection with conditioned stimulus 
intensity (p. 119). Different groups of subjects are trained initially 
under two or more different levels of motivation. Then, at some point, 
each of these groups is subdivided. Some of the subjects continue under 
the origiual drive condition ; others switch to the alternative drive levels. 
For the simplest situation, where only two drive levels are involved, this 
procedure generates the 2x2 factorial design represented in Table 28, 


TABLE 28 

Experimental Desicn for Separating Effects of Drive on 
Performance from Effects of Drive on Learninc 


Later Teat Drive 
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Means reflecting performance differences 



in which the rows represent the conditions of original training and the 
columns represent conditions after drive levels have been switched for 
some of the subjects. As with the CS-intensity example, differences in 
the column means represent performance differences, and differences in 
the row means reflect learning differences; that is, differences due to the 
original training drives are now reflected with drives equated. 

There are six points which should be emphasized with respect to this 
diagram: (1) It is to be noted that the measures in the table are those 
obtained in the second phase of the experiment. (2) This second phase 
has taken a number of different forms in different studies. The test has 
sometimes been in extinction, sometimes in recall, and sometimes in a 
series of further reinforced trials. (3) The measures of behavior em- 
ployed have included resistance to extinction, running time and its 
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various derivatives, and errors in a maze. (4) Interest in these experi- 
ments has centered about the question of the differences in the row means, 
which reflect learning differences. The fact that the column (perform- 
ance) means differ in almost all experiments of this sort merely reflects 
the point, on which there is no disagreement, that motives influence be- 
havior. (5) Not all of the experiments attempting to answer this question 
have employed a complete factorial design. Particularly in the earlier 
studies, the procedure sometimes was to train subjects under a variety 
of drive levels and then to test with drive equated for all groups. This 
creates problems because shifting from a high drive to an intermediate 
one may not be the same as shifting from a low drive to an intermediate 
one. (6) Especially in these latter experiments there sometimes seems 
to be a residual effect of the earlier drive level which enters and compli- 
cates the interpretation of the results in the second phase of the experi- 
ment (Deese and Carpenter, 1951). 

The theoretical framework for experiments on the effects of drive on 
learning has, in recent times, most often been provided by Hull’s (1943) 
theory developed in Principles of Behavior. In that book, Hull formalized 
the distinction between learning and performance, and he introduced 
constructs corresponding to each. The learning construct, habit (sHr), 
was hypothesized to be a function of such variables as the number of 
reinforced trials, the amount of reinforcement, delay of reinforcement, 
and stimulus-response asynclironism. Drive was not included. It entered 
as a multiplier of habits in the determination of the strength of the 
performance variable, excitatory potential ( s E n ). Thus, Hull made drive 
a performance variable and proposed that it had no effect upon learning. 

Experiments which may be regarded as tests of this theory have pro- 
duced results both for and against Hull’s hypothesis. That is, some have 
obtained no performance differences attributable to drive at the time of 


learning. This is the result predicted by Hull. Others have obtained such 
differences. Those in the latter category have usually found that higher 
dri\c at the time of original learning leads to better learning; but a few 
have found learning to be most efficient at some intermediate or lower 
level of motivation (Finan, 1940; Reynolds, 1949). 

A tabuhu summon of some of the results obtained in experiments on 
the question under discussion appears in Table 29. Careful inspection of 
this table wdl show that the number of experiments which have obtained 
j>OMti\i results is just about equal to the number obtaining negative re- 
sults. Moreover, there seems to be no breakdown by procedure, type of 
dnve. or response measure which provides a certain indication of the 
fmuor. rospnns.ble f or the discrepant results. The only discernible tend- 
ency is air experiments involving complex mazes to show an effect of 

''HZ All except one of these 

sonic ■■ Ill'll'.,;, H miter. and Unable, 1DM) have obtained such results. 
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At the most general level it would seem that the condition determining 
•whether drive apparently influences learning in a given experiment is 
not among those reflected in the table. What this variable is (if, indeed, 
there is only one) remains a matter of speculation. The most likely possi- 
bility is that certain detailed features of the behavior occurring in 
original training may be involved. Campbell and Kraeling were the first 
to call attention to this possibility. 

As is shown in Table 29, Campbell and Kraeling obtained evidence for 
an effect of training drive upon learning. In their discussion of this fact, 
these investigators mentioned that it was their impression that the rats 
learning under low drive behaved differently from those behaving under 
high drive. Specifically, the subjects under high drive appeared to have 
acquired a marked readiness to leave the starting box, which the low- 
drive group had not. In behavioral terms the high-drive animals tended 
to maintain an orientation toward the door leading from the starting 
chamber and to leave the box as soon as it was opened. The low-drive 
animals, by contrast, developed no such tendency. If this interpretation 
is correct, two points follow: The first is that drive will show an effect 
upon learning whenever the training situation evokes different behavior 
under different drive levels. On mainly common sense grounds, complex 
situations might be expected to maximize such effects. In the complex 
maze, for example, exploratory tendencies might dominate the behavior 
of low-drive subjects, with the result that such subjects would learn 
somewhat different habits from high-drive subjects who explored less. 
In this connection, it has already been noted that the use of complex 
mazes, more often than any other procedure, tends to yield results indi- 
cating an effect of drive on learning. Moreover, the one experiment pro- 
ducing the opposite result (Hillman, Hunter, and Kimble, 1954) differed 
from the others in one very important respect : the rats used as subjects 
were blind. This might very well eliminate exploratory tendencies if, as 
seems to be the ease, such tendencies depend upon visual stimulation. In 
support of this interpretation, Hillman, Hunter, and Kimble report that 
there were no differences in error scores during phase 1 of the experiment, 
suggesting that the animals’ behavior was similar in both drive groups. 

The same line of reasoning also leads to the conclusion that the effect 
of drive on learning, when such an effect occurs, would not always have 
to be uniformly increasing. If some intermediate drive level elicited be- 
havior better adapted to the requirements of the situation, then the most 
efficient learning would occur under these conditions. This sort of argu- 
ment, already developed in connection with the energizing effect of mo- 
tives on behavior, will not be amplified upon further, except to point out 
that, in a highly speculative and ad hoc way, the results obtained by 
Finan and by Reynolds can be made to fit such a scheme. 

The second major point suggested by the emphasis upon drive-produced 
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various derivatives, and errors in a maze. (4) Interest m these experi- 
ments lias centered about the question of the differences in the row means, 
which reflect learning differences. The fact that the column (perform- 
ance) means differ in almost all experiments of this sort merely reflects 
the point, on which there is no disagreement, that motives influence be- 
havior. (5) Not all of the experiments attempting to answer this question 
have employed a complete factorial design. Particularly in the earlier 
studies, the procedure sometimes was to train subjects under a variety 
of drive levels and then to test with drive equated for all groups. This 
creates problems because shifting from a high drive to an intermediate 
one may not be the same as shifting from a low drive to an intermediate 
one. (6) Especially in these latter experiments there sometimes seems 
to be a residual effect of the earlier drive level which enters and compli- 
cates the interpretation of the results in the second phase of the experi- 
ment (Deese and Carpenter, 1951). 

The theoretical framework for experiments on the effects of drive on 
learning has, in recent times, most often been provided by Hull’s (1943) 
theory developed in Principles of Behavior. In that book, Hull formalized 
the distinction between learning and performance, and he introduced 
constructs corresponding to each. The learning construct, habit ( S H B ), 
was hypothesized to be a function of such variables as the number of 
reinforced trials, the amount of reinforcement, delay of reinforcement, 
and stimulus-response asvnchronism. Drive was not included. It entered 
as a multiplier of habits in the determination of the strength of the 
performance variable, excitatory potential ( S E U ). Thus, Hull made drive 
a performance variable and proposed that it had no effect upon learning. 

Experiments which may be regarded as tests of this theory have pro- 
duced results both for and against Hull’s hypothesis. That is, some have 
obtained no performance differences attributable to drive at the time of 


learning. This is the result predicted by Hull. Others have obtained such 
differences. Those in the latter category have usually found that higher 
dri\e at the time of original learning leads to better learning; but a few 
lm\e found learning to be most efficient at some intermediate or lower 
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At the most general level it would seem that the condition determining 
whether drive apparently influences learning in a given experiment is 
not among those reflected in the table. What this variable is (if, indeed, 
there is only one) remains a matter of speculation. The most likely possi- 
bility is that certain detailed features of the behavior occurring in 
original training may be involved. Campbell and Kraeling were the first 
to call attention to this possibility. 

As is shown in Table 29, Campbell and Kraeling obtained evidence for 
an effect of training drive upon learning. In their discussion of this fact, 
these investigators mentioned that it was their impression that the rats 
learning under low drive behaved differently from those behaving under 
high drive. Specifically, the subjects under high drive appeared to have 
acquired a marked readiness to leave the starting box, which the low- 
drive group had not. In behavioral terms the high-drive animals tended 
to maintain an orientation toward the door leading from the starting 
chamber and to leave the box as soon as it was opened. The low-drive 
animals, by contrast, developed no such tendency. If this interpretation 
is correct, two points follow : The first is that drive will show an effect 
Upon learning whenever the training situation evokes different behavior 
under different drive levels. On mainly common sense grounds, complex 
situations might be expected to maximize such effects. In the complex 
maze, for example, exploratory tendencies might dominate the behavior 
of low-drive subjects, with the result that such subjects would learn 
somewhat different habits from high-drive subjects who explored less. 
In this connection, it has already been noted that the use of complex 
mazes, more often than any other procedure, tends to yield results indi- 
cating an effect of drive on learning. Moreover, the one experiment pro- 
ducing the opposite result (Hillman, Hunter, and Kimble, 1954) differed 
from the others in one very important respect : the rats used as subjects 
were blind. This might very well eliminate exploratory tendencies if, as 
seems to be the case, such tendencies depend upon visual stimulation. In 
support of this interpretation, Hillman, Hunter, and Kimble report that 
there were no differences in error scores during phase 1 of the experiment, 
suggesting that the animals’ behavior was similar in both drive groups. 

The same line of reasoning also leads to the conclusion that the effect 
of drive on learning, when such an effect occurs, would not always have 
to be uniformly increasing. If some intermediate drive level elicited be- 
havior better adapted to the requirements of the situation, then the most 
efficient learning would occur under these conditions. This sort of argu- 
ment, already developed in connection with the energizing effect of mo- 
tives on behavior, will not be amplified upon further, except to point out 
that, in a highly speculative and ad hoc way, the results obtained by 
Finan and by Reynolds can be made to fit such a scheme. 

The second major point suggested by the emphasis upon drive-produced 
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behavior is more basic. Insofar as Hull’s theory is concerned, the implica- 
tion is that the theory has not been refuted. Such a refutation would 
come from an experiment in which (1) drive at the time of learning were 
shown to have an effect later, with drives equated, and (2) the responses 
elicited during original training were shown to he exactly the same for 
the two groups. Thus far, no experiment meeting the second of these 
requirements has been reported. The one attempt at such an experiment 
failed because the first condition was not met. In this experiment (R. A. 
King, 1959), animals were trained in a straight runway, at first under 
12 and 60 hours of food deprivation; then, in extinction, drives were 
reversed for half the animals. It will be noted that this is exactly the set 
of conditions used by Campbell and Kraeling in their experiment which 
obtained an effect of drive on learning. To evaluate the importance of 
competing responses in such experiments, King used two different re- 
sponse measures. One measure was running time in the usual sense ; the 
other was running time with the time spent performing competing re- 
sponses subtracted out. The expectation was that the first measure would 
yield results which seemed to show an effect of drive on learning, and 
that the second measure would not, thus assigning the influence of drive 
to the mechanism of competing responses. The results, however, failed 
to show an effect of drive on learning in either measure. It thus appears 
that an important experiment remains to be done. 


DRIVE STIMULI AND LEARNING 

The concept of drive stimulus was introduced in a preliminary, 
methodological, way earlier in this chapter. The main concern of this 
section will be with experimental studies of its role in the learning 
process. In general, such insestigations have been planned on the assump- 
tion that dri\ c stimuli will influence behavior in the same way as other 
stimuli do, and that such processes as discrimination and generalization 
apply to drive stimuli (Webb, 1955). 


Discrimination Among Drive Stimuli 

The elmsiea! experiment demonstrating tlie discriminability of drive 
st.mnl, teas done I,,- Hull (1933). Il 0 ran rats in a simple mate, hungry 
on «,„e days and tlnrsty on others. On the hunger days one turn in the 
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goal box. Leeper (1935) showed that the problem could be made some- 
what easier by using separate goal boxes and very much easier by other 
procedural alterations favoring the discriminability of the two reinforce- 
ment locations. From the nature of his results, Leeper argued that what 
was important in such learning was the development of cognitions con- 
cerning the location of reward. Seeman and H. 'Williams (1952), on the 
other hand, have proposed that the Hullian concepts of secondary rein- 
forcement and T 0 can handle Leeper ’s results. They performed an experi- 
ment comparing the speed of learning to respond in terms of drive state 
when the final path in the maze and the goal box were the same for the 
two drives and when they were different. To produce this difference See- 
man and Williams used a maze in which the final common path was 
always in the same physical location. For a control group of rats, it was 
also perceptually the same whether the drive was hunger or thirst. For an 
experimental group, however, this path and the goal box were black on 
hunger days and white on thirst days. The experimental subjects learned 
much better than the control subjects, supporting the argument for 
secondary reinforcement. 

Still other conditions related to the speed of learning a discrimination 
between drive stimuli have been described by Bolles and Petrinovich 
(1954). These investigators called attention to the following differences 
between the procedures used to study discrimination between drives and 
procedures employed in the investigation of discriminations between ex- 
ternal environmental cues: (1) In the ordinary discrimination-learning 
procedure, the stimuli to be discriminated are presented rather suddenly, 
for example, at the choice point in a maze. In drive discrimination, the 
intensity of the stimuli increases gradually, and these stimuli are present 
outside the choice situation. (2) In the ordinary discrimination situation, 
the discriminanda are absent when the animal is taken from the appara- 
tus. In drive discrimination studies, they are still present. (3) In ordi- 
nary discrimination learning, tlie stimuli are usually physically separate. 

In drive discrimination studies Lunger and thirst Lave probably always 
been present at the same time. Bolles and Petrinovich performed an experi- 
ment in which animals learned to go one way in a modified T-maze when 
hungry and the other way when thirsty. In order to reduce the differences 
described above to a practicable minimum, they employed the following 
special procedures: (a) The animal’s home cage was the choice point of 
the maze, so that the drives to be discriminated developed in the choice 
situation, (b) The animals were run one trial a day and were allowed 30 
minutes or more in the correct goal box with food or water so that the 
drive stimuli would have been eliminated when the subject was removed 
from the goal box at the end of the trial. ( c ) A special deprivation 
schedule described by Verplanck and Hayes (1953) was used to separate 
hunger and thirst as sharply as possible. Under these conditions the 
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subjects required an average o£ only 4.5 trials to reach a criterion of 9 

correct responses in 10 trials. _ . 

There have been very few experiments in which animals were required 
to learn to discriminate between the stimuli provided by drives other 
than hunger and thirst. In one example of such a study, however, Amsel 
and McDonnell (1951) were moderately successful in training rats to go 
one way in a T-maze to escape shock and the opposite way to escape the 
stimuli associated with shock. This suggests that the motives of escape 
from pain and fear can be distinguished in somewhat the same way as 
hunger and thirst. 

With the discriminability of the stimuli provided by different drives 
demonstrated, the next question to arise is whether different intensities 
of the same drive are also discriminable. There is evidence (Bloomberg 
and Webb, 1949 ; Jenkins and Hanratty, 1949) to indicate that they are. 
Jenkins and Hanratty, for example, ran rats in a single-unit T-maze 
alternately under 11.5 and 47.5 hours of deprivation, with a right turn 
correct under one drive level and a left turn correct under the other. 
Tlie animals learned this discrimination in a matter of 100 trials or so. 
At the end of the experiment the subjects were tested under 4 and 72 
hours of deprivation, with the result that they responded under the lower 
drive as they had at 11.5 hours of deprivation and under the higher 
drive as they had at 47.5 hours of deprivation. In a somewhat novel way, 
this shows the generalization of a habit along a dimension of drive stimu- 
lus intensity. 

The fact that rats could learn to discriminate these particular drive 
intensities has a bearing on a related problem. Heron (1949) obtained, 
as had Hull and Leeper, a discrimination between hunger and thirst. A 
scries of control tests, however, suggested that the subjects in this experi- 
ment had not discriminated the distinctive drive stimuli, but may have 
responded to relative fullness of the stomach which could have differed 
under the two drive conditions. The Jenkins and Hanratty demonstra- 
tion casts doubt upon this as a general interpretation, because, under 
both of their conditions, the stomach would have been completely’ empty . 
See also Bailey and Porter, 1955. 


The Selective and Nonselective Theories of Drive 
Stimulation 


In interpreting the results of Ins early study, Hull had proposed th 
learning to discriminate among drive stimuli is possible on the basis 
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Some years later, Kendler (1946) set out to test the validity of Hull’s 
theory by the following experiment. Rats, simultaneously hungry and 
thirsty were run in a T-maze with essentially forced trials which led 
them sometimes in one direction to water and an equal number of times 
in the other direction to food. Specifically, there were four trials a day. 
The first and third trials were free; hut on the second and fourth trials, 
the animal was forced to go in a direction opposite to that on the pre- 
ceding trial. In this manner, experience with each maze path and each 
reward was equalized. Following this training there was a series of four 
tests. On some days the animal was only hungry; on other days it was 
only thirsty. The point in question was whether the animal would now 
behave appropriately, going to water on thirst days and to food on 
hunger days. 

Hull’s theory, strictly applied, predicts that no such learning could 
occur because the hunger and thirst stimuli had been simultaneously 
present and should have been conditioned to the right and left responses in 
approximately equal degrees. When only one drive state was present there 
would be no stronger tendency to go in one direction than in the other. 
The behavior of the rats gave no support for such a hypothesis. On thirst 
days they ran correctly to water on 98 per cent of their trials ; on hunger 
days they ran to food on 73 per cent of the trials. In order to explain this 
result, Kendler proposed what he called a selective principle of associa- 
tion of drive stimuli which stated that “only those drive stimuli which 
are themselves reduced become connected to the rewarded response.” 
(Kendler, 1946, p. 220). This is to be contrasted with Hull's non* 
selective principle according to which all stimuli become connected to the 
rewarded response. 

Although some experiments have produced results modestly supporting 
the selective association principle (Kendler, 1949; Kendler and Law, 
1950), the most convincing studies have obtained negative results. With 
an occasional exception (Woodbury and Wilder, 1954 ; Porter and N. E. 
Miller, 1957) these studies have adopted a strategy which differs some- 
what from that of Kendler ’s original experiment. Where Kendler, follow- 
ing the lead of Hull and Leeper, had tested the effectiveness of the stimuli 
provided by relevant drives, more recent investigations have made the 
critical cues the stimuli provided by irrelevant drives. In all of these 
investigations animals have learned different responses in the presence 
of hunger and thirst stimuli, but the responses led to the satisfaction of 
another motive through anxiety reduction (Amsel, 1949) or light avoid- 
ance (Winniclc, 1950; Levine, 1953; Bailey, 1955). 

Bailey used five groups of 10 rats each in an escape-training procedure 
in which pushing one of two panels would turn off a noxious light. The 
correct panel differed from day to day in a manner which depended upon 
the particular group. For three groups, variations in drive provided the 
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cue stimuli; for the two other groups, presence or absence of a tone 
served the same purpose. Specifically, the five conditions required the 
subjects to discriminate between (a) hunger and thirst, (6) hunger and 
satiation, (c) thirst and satiation, (d) presence or absence of a tone 
sounded after the animals were placed in the apparatus (short-sound 
group), and (e) presence or absence of a tone which, when used, was 
continuously present over a long period of time (long-sound group). All 
the animals except those in the long-sound group learned to respond 
correctly. This demonstrated that drive stimuli can serve as cues although 
they are not reduced by the rewarded response, thus providing strong 
evidence against the selective association principle. Moreover, the drive 
stimuli were about as effective as cues as the short tone. Bailey attributed 
the fact that long tone provided so poor a cue to the operation of a prin- 
ciple suggested by Hull, that unchanging (static) stimuli are less effective 
cues tlian changing (dynamic) ones (Hull, 1943, p. 207). Since hunger 
and thirst stimuli show periodic fluctuations in intensity, they would be 
more dynamic than the tone which was continuously present at a constant 
level. 


Generalization on a Dimension of Drive Stimulation 

Another direct implication of the drive-stimulus concept is that alter- 
ations in drive will lead to a relative loss of response strength because 
changing drive changes a part of the stimulus complex to which the 
response had been conditioned. Data which support this prediction have 
come from a variety of sources. Heathers and Arakelian (1941) found 
that a bar-pressing response extinguished more rapidly if two extinction 
sessions involved the same degree of hunger than if the two sessions in- 
volved different drive levels. Barry (1958) found that changing drives 
led to less rapid running early in extinction, although, later in extinction, 
the effect of changing drives was the opposite. Elliott (1929), in a famous 
study, showed that changing drives midway in the course of maze leam- 
imr produced a temporary disruption of behavior. In this experiment a 
control group of 32 thirsty rats learned a maze for water. An experi- 
mental group began with the thirst-water condition but on the tenth of 
20 trials were shifted to a hunger-food regime. The effect of this manipu- 
lator is shown in Figure 13.7 for error scores. There was a similar but 
relatively smaller effect on running time. Although the concept of drive 
stimulus is not directly involved, it is worth mentioning that alterations 
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FIGURE 13.7. Learning curves showing effect of a changed reward vp on the 
performance of ratB in a maze. The control group was rewarded with sun- 
flower seed {com the beginning. The experimental group received a wet mash 
as reward for the firBt nine days and sunflower seeds after that. Note that this 
change disrupted the rats' behavior and produced a response level which was 
much inferior to that of the control group. (Elliott, 1928). 

Since, theoretically, the mechanism by which alterations in drive dis- 
rupt learning is that of stimulus generalization, it should be possible to 
demonstrate a generalization gradient along a dimension of drive stimu- 
lus intensity. A response conditioned at one drive level and tested at 
Several others should show a diminution in strength, and the amount of 
this diminution should increase with the difference between training and 
test drives. Although the prediction is a straightforward one, testing it 
proves quite difficult. The most important consequence of changing drive 
(which is essential in the performance of the necessary experiment) is 
to increase or decrease the vigor of response. Raising drive level increases 
response strength, and lowering it decreases response strength. Moreover, 
these effects ordinarily are of greater magnitude than any that can be 
produced merely by changing drive stimulation. Taking this fact into 
account, however, it is possible to make a general prediction about the 
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form of the generalization gradient along a drive-stimulus dimension. 

To illustrate, consider the case of a rat conditioned to press a bar to 
obtain food at an intermediate drive level, say 24 hours of deprivation, 
and tested at the training drive and several others, for example, 3, 12, 
48, and 72 hours of deprivation. Tests at 3 and 12 hours, which are below 
the level of deprivation during training, may be expected to lead to 
lowered response strength for two reasons: {a) drive is lower, and the 
response will be weakened for this reason; (b) the drive stimuli are 
different, and response strength should be reduced as a result of gen- 
eralization decrement. Tests at 48 and 72 hours, which are above the 
level of deprivation during training, pit these two effects against each 
other. The increased deprivation should raise response strength ; the 
altered stimulus situation should lower it. These considerations lead to 
the prediction that the generalization gradient obtained when drive is 
lowered will be steeper than that obtained when drive is increased (Meehl 
and MacCorquodale, 1948). 

The only extensive evidence on this point is from an experiment by 
Yamaguehi (1952) in which rats were trained to press the bar in a 
Skinner box at one drive level and tested at the same drive level and 
certain others. Table 30 presents a set of results in terms of reaction 


TABLE 30 


Median Latencies of Bar-Press Response During Testing 

(Numbers in bold type are for conditions in which test 


and training conditions 

were the same. 

Yamaguehi, 1952) 


Hours of Hunger During Testing 

Hours of Hunger 

During Training 


3 

24 

48 

72 

3 

.97 

1.05 

1.19 

1.65 

24 

.93 

.85 

.90 

.96 

48 

.88 

.89 

.87 

.95 

72 

.97 

.83 

.89 

.85 


latency. The generalization gradients for each training drive level are 
represented by the numbers in the columns. Close inspection of these 
data will show that the expected relationships occur. For the subjects 
trained at 3 hours of deprivation, the main effect of chanrinu drive is to 
improve performance. It is only with the very largest change in drive 
( it-hour condition) that the effect, of generalization decrement is suffi- 
ewnt to overcome the opposite effect of raising drive level. For the other 
rond, turns U „ to be noted that, with a single exception, the most rapid 

™ rt,on ? m,rS at th(> <lrivp lpvc ’l in training. And, as expected ?ie 
response strength (increase in latency) is much -reatet with 
!>.v <-r«-d drive than with increased drive * c G1 * lth 
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The Intensity of Drive Stimulation and tlie Range 
of Behavior 

There is, as we have seen, considerable evidence that one of the effects 
of increasing drive is to eliminate competing responses. It becomes impor- 
tant at this point to ask why this should be. Classical S-R and cognitive 
theory suggest two rather different-sounding answers. S-R theorists have 
held that any response in the process of being learned competes with alter- 
native responses, and that increases in drive increase the differences 
among the strengths of the various response tendencies. Thus, when the 
response being measured is dominant, increasing drive will eliminate com- 
peting responses because the correct response is strengthened relative to 
them. In most instrumental conditioning situations (for example, Skinner 
box and runway) the apparatus is arranged so as to make the measured 
response dominant either from the beginning or very early in learning. 
Thus, it follows that increased drive will increase the strength of the 
dominant response and eliminate its competitors. 

The cognitive theorist, on the other hand, reasons as follows. Increases 
in drive strength make the motivational situation so important that it 
dominates the organism’s behavior, and the animal responds to its drive 
state (So) to the exclusion of everything else. Tolman (1948) has put this 
argument most clearly. He referred to heightened drive as a narrower of 
cognitive maps, meaning by this that intense motivation tends to limit 
the animal’s perceptions and to restrict them to stimuli directly involved 
in the satisfaction of the drive in question. Other events in the environ- 
ment, having no significance, are not noticed. 

Bruner, Matter, and Papanek (1955) performed an experiment which 
was designed as a direct test of Tolman ’s theory. The procedure involved 
three stages. In tlie first stage, rats learned a black-white discrimination 
in a linear maze with four choice points. In this stage of the experiment, 
the position of the black and white stimuli were randomly varied from 
choice point to choice point. Stage two was a continuation of tlie black- 
white discrimination problem, but in this series of trials, the positive 
(for example, black) stimulus was alternately on the right and left at 
successive choice points for the experimental groups. For a series of 
control groups the positive cue continued to be randomly right or left. 

In the third and final phase of the experiment all subjects were required 
to learn the single-alternation pattern introduced incidentally for the 
experimental group in phase two of the experiment. Gray doors were 
substituted for the black and white ones, and the only available cue mark- 
ing the correct choice was position. 

Throughout the experiment, some of the animals had been kept 32- 
hours hungry; others had been 36-hours hungry. The general prediction 
made from Tolman ’s theory was that the 32-hoar animals would learn 
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the single alternation faster than the 36-honr animals in stage three of the 
experiment, for the following reason: During the second phase of the ex- 
periment, being less dominated by their drive state, , the 12-hour animals 
would have tended to perceive the single-alternation sequence of cor- 
rect choices and would he able to transfer this information to the later 
test phase of the experiment. The results supported this prediction, al- 
though the differences were not large. It seems possible that the 12-hour 
deprivation condition may have made the subjects too hungry for the 
effect to appear in maximal proportions. A group of subjects only about 
2-liours hungry would have provided a clearer differentiation of the high 
and low drive levels. 

LEARNED MODIFICATIONS OF MOTIVES 

Up to this point our discussion has been focussed on learning, and 
motivation has been viewed as a contributor to it. It must be recognized, 
however, that the reverse perspective is possible. Motives, themselves, 
depend upon learning; and, especially in recent years, there has been 
an increasing interest in the topic of acquired drive or secondary 
motivation. 


Instinct and Secondary Motivation 

IVatson’s (1925) behaviorism, as is well-known, represented a break 
from previous tradition, in its adoption of an extreme environmentalism. 
Man, in the eyes of the environmentalist, is born with a limited number 
of unconditioned reflexes which, by conditioning and chaining, eventu- 
ally produce the elaborate set of skills which are typical of the adult. 
The hehaviorist’s view of motivation was also very similar to this. Besides 
his innate reflexes, the infant possesses, Watson thought, a few uncondi- 
tioned emotional reactions to certain specific stimuli, and the richness of 
adult emotional experience develops, by conditioning, out of these primi- 
tive raw materials. As a sort of case demonstration, Watson and Rayner 
(1920) did the experiment which is now a classic. A rat (CS) was shown 
to a child. Albert, simultaneously with a loud and unpleasant sound 
(PCS) produced by striking a steel bar behind the child’s head The 
vound caused the child to cry (BCR). After a few repetitions, the child 
cried at the sicht of the rat (C'R) and this fear generalized to other furry 
such as a fur neckpiece or a Santa Claus mask. Attempts to 
rep.-at the V at sun and Rayner study (English. 1929; Bre<unan 1934) 
v-re not alv.avs sucee-ful ; and Valentine (1930) made the cogent point 
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such as a pair of opera glasses. But, in spite of the limitations suggested 
by these later studies, the effect of the Watson and Rayner study upon 
the immediately subsequent history of psychology was very great. It 
demonstrated a general methodology which has been used in many more 
recent fear conditioning studies. It also suggested that what have been 
called instinctive reactions may be responses conditioned to previously 
neutral stimuli. With time this last point has undergone a subtle trans- 
formation, with the result that one seldom hears of the eonditionability 
of instinctive reactions. Instead, the point has become that the responses 
once thought to be instinctive are actually learned. 

Whatever the merits of this way of phrasing it, the reformulation 
precludes the asking of an interesting question : are instinctive reactions, 
strictly conceived, conditionable? Recent years have seen the reacceptance 
of the instinct concept by psychologists, mainly as a result of the work 
of a group of zoologists of whom the best known are Lorenz (1952) and 
Tinbergen (1951). These two men in particular have advocated theories 
which have certain parallels to psychological theory. They have also 
made behavioral studies which have important psychological implications. 

The instinct concept is central in these theories. Instinctive behavior is 
defined as an unlearned, relatively stereotyped act or series of acts (of 
which the most elementary seem almost exactly the same as unconditioned 
responses) which is tied to some specific environmental stimulus (paral- 
leling the psychological concept of UCS). The energy (psychologically, 
drive) for instinctive behavior is thought of as existing in the nervous 
system as action specific energy. Instinctive behavior is elicited by a 
highly specific stimulus, or releaser, which initiates the action of a neuro- 
logical innate releasing mechanism. Except in the presence of a releaser, 
the action specific energy is held in check or blocked, so that the instinc- 
tive act ordinarily occurs only to the biologically appropriate stimulus. 
Occasionally it does break through this inhibiting barrier and appears 
in response to the wrong stimulus, in which case the response is referred 
to as displacement activity, or to no apparent stimulus at all, in which 
case it is referred to as vacuum activity. As the parenthetical expressions 
in the description above indicate, instinct theory contains an obvious 
parallel to the concepts of drive, unconditioned stimulus, and uncondi- 
tioned response. In some of the literature the distinction between instru- 
mental and consummatory behavior made previously in this chapter 
also appears (Thorpe, 3954). 

Although tlie emphasis in the recent instinct literature is upon un- 
learned acts, the importance of learning is not only admitted, hut is of 
prime importance in connection with at least two concepts, instinct’ 
training interlocking and imprinting. The former of these refers to the 
fact that sequences of instinctive acts (for example, the complicated 
series of responses involved in the flight of birds) arc apparently hound 
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together by the effects of stimuli which have acquired significance as a 
result of learning. The latter concept, imprinting, refers to the emotional 
attachments which the infants of many species seem to invest m the first 

large, moving object they see. _ ... 

Although the description of such learning is seldom put in conditioning 
terms, it seems possible that the conditioning of released responses can 
occur. Lorenz (1952) describes what is probably a case in point in the 
behavior of the jackdaw. This bird instinctively attacks any animal, in- 
cluding man, carrying a dark limp object of appropriate size, such as a 
wounded jackdaw or a pair of dark swimming trunks. Lorenz himself 
was attacked on several occasions when carrying a pair of black swim- 
ming trunks and reports that, afterward, the jackdaws would attack 
him, even without the trunks. This seems to be a clear case of condition- 
ing. If it turns out to be fairly dependable, it will be possible to bring 
the phenomenon under a degree of laboratory control which has so far 
not been typical of studies in this area. 

The phenomenon of imprinting is an exception to this last statement. 
Several investigators have done laboratory work on imprinting. Jaynes 
(1956, 1957), for example, has performed a series of studies showing that 
baby chicks can readily be imprinted by making the first large moving 
object they see a colored cube or cylinder. After such experience, the 
chicks follow the object and remain near it much as they would if it 
were their mother. Moreover, Jaynes has also shown that such phenomena 
as acquisition and generalization apply to imprinted behavior, suggesting 
that there is a close relationship between it and other learning. There 
remains the question whether what is learned is a motive. The answer 
will come in the form of experiments to determine whether the conven- 
tional criteria of motivation apply. Some observations of Hess (1958) 
suggest that they will : ducklings will overcome obstacles in order to get 
to a decoy on which they have been imprinted. 


Fear as an Acquirable Drive 

It has been necessary at previous points in this book to explain the 
mechanism by which a number of different theorists believe that fear 
develops (pp. 267-277). Briefly, this theory is that stimuli associated 
with painful events come, by a process of classical conditioning, to evoke 
fear. The status of fear as a motive is then inferred from the’fact thal 
it has the same properties as other motives, those of providing the bash 
for learning and of influencing the vigor of behavior. 

Effects in this latter category are of two opposite sorts. Sometimes tin 
r r r aao " of * stimulus augments a response- sometimes i 

m uure W, demonstrated the facilitating effect of fear in the follow 



MOTIVATION AND LEARNING 


427 



FIGURE 13.8. Stabilimeter for measuring startle response. Activity in the 
confinement box is registered in graphic form on the polygraph (Brown, 
Kalish, and Farber, 1951). 



FIGURE 13,9. Augmentation of the startle response to n load sound by the 
addition of a conditioned fear stimulus. The increasing trend in the magni- 
tude of the startle response for the experimental group shows the acquisition 
of fear ns o result of continued pairing of the CS with shock (Brown, KaHili, 
and Farber, 1951), 
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ing •way. An experimental group of rats received four days of training 
■vatli 10 trials each day. Seven of these trials consisted of the pairing of 
a CS (light and bnzzer) -with shock at a 3-seeond interval. On the other 
three trials the shock was omitted and the sound of a pop-gun substi- 
tuted for it. Ordinarily the sound of the pop-gun produced a startle 
response. Presenting the fear stimulus prior to the startle stimulus in- 
creased the magnitude of the startle. This effect increased during the 
experiment as is shown in Figure 13.9. A control group which received 
the same number of CS presentations and the same number of shock 
trials showed no systematic trend in the magnitude of the startle re- 
sponse. Following the series of days represented in Figure 13.9 there 
were three extinction days in which the CS and the sound of the pop-gun 
were presented together but the rats were never shocked again. Results 
obtained in this part of the study exhibited an extinction trend within 
days and a spontaneous recovery between days, showing that these 
processes, as well as acquisition, apply to the conditioned fear drive. 

A series of studies beginning with one by Estes and Skinner (1941) 
has shown that a fear-eliciting stimulus sometimes has quite a different 
effect upon behavior, that of producing an interruption of ongoing 
activity. H. F. Hunt, Brady, and their associates have reported a num- 
ber of demonstrations of this fact in a series of studies more centrally 
concerned with the effect of electroconvulsive shock upon fear than with 
the properties of fear itself. The procedure involved in a typical study 
is as follows : Rats are first trained in a Skinner box to press a bar for 
water on a variable-interval schedule. When this response has been 
thoroughly established, the fear conditioning begins. This conditioning 
process consists of the presentation of a 16-per-second click for a period 
of three minutes, followed by a brief, but fairly strong, shock as the 
clicking sound terminates. Initially the clicker has no effect upon the 
animal’s behavior, but, with a few presentations, it acquires the ability 
to inhibit bar pressing completely. During the three-minute CS presenta- 
tion. the animal typically crouches in the apparatus and often defecates 
or urinates. 1 
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The fact that conditioned fears have different, and opposite, effects 
upon behavior poses no particular problem of interpretation. We have 
seen in connection with motives in general that their effects cannot be 
predicted without taking into account the responses elicited by the drive- 
establishing operation and the relationship of these responses to the 
responses they influence. The same sort of reasoning applies here. A 
series of experiments by Amsel and his associates may be cited to make 
this point. These studies involve the effect of fear upon the consummatory 
response of drinking. In the first investigation in this series (Amsel and 
Maltzinan, 1950) rats were first trained to drink for 10 minutes each 
day in individual drinking cages. After the animals’ intake had stabilized, 
they were given an electric shock just before the drinking test in a situ- 
ation which was different from that in which they were tested. In the 
subsequent test there was an increase in the amount the animals drank. 
Amsel and Maltzman reasoned that the shock had increased the animals' 
fear and that this fear had served to energize the dominant response in 
the situation, namely, drinking. In this interpretation the fact that the 
shock occurred outside the drinking situation is of first importance. 
Later studies demonstrated that giving the shock in the drinking cages, 
or in compartments like the drinking cages, interfered with consumma- 
tory behavior (Amsel, 1950b) and that the degree of interference de- 
pended upon the degree of similarity between the drinking situation and 
the shock situation (Amsel and Cole, 1953). Together these results suggest 
that the shock leads to the conditioning of responses incompatible with 
drinking to the cues in the shock situation. If the shock and drinking 
situations are similar, these responses generalize to the latter and inter- 
fere with consummatory behavior. If they are quite different, all that 
generalizes is a heightened degree of emotionality which increases intake. 2 

The ability of a conditioned fear to motivate new learning may be 
illustrated by the study of J. S. Brown and Jacobs (1949). These investi- 
gators (experiment II) used two groups of 8 rats each and subjected the 
experimental group to a fear-conditioned procedure which consisted of 
the pairing of a CS (tone and light) with shock for 22 trials. The CS 
lasted for a total of 9 seconds on each trial, and the shock was presented 
during the last 6 seconds of this period. This training took place in one 
side of a two-compartment box, access to the other side having been pre- 
vented by a door. To test for the drive value of the CS, this door was 
raised and the subjects received 40 trials in which they could escape 
from the tone and light by jumping a little hurdle into the other side of 
the box. Results of the experiment appear in Figure 13.10 for the experi- 
mental group and for a control group, which had received no fear eondi- 

■Hoisted studies are those of Lichtenstein (1930a, 1050b), Siegel and Brantley 
and Siege! and Siegel (1940). 



430 


CONDITIONING AND LEARNING 



SUCCESSIVE BLOCKS OF FIVE LEARNING TRIALS 

FIGURE 13.10. Learning motivated by the presentation of a feared stimulus 
and reinforced by the cessation of the same stimulus (Brown and Jacobs, 
1949). 

tioning. It is evident that the experimental animals learned and the 
controls did not. 


Ollier Acquired Drives 

Although the majority of the work on acquired drives has involved 
fear, there has been some investigation of other similarly developed 
motives. The motives with which such further investigation have dealt 
have comprised three clusters: (1) incentive motivation, (2) the “ex- 
ternalized” hunger drive, and (3) what Tolman called social techniques. 
"We have considered the first of these previously (p. 170) and, so, will 
discuss only the second two here. 

1. The “Extrmalization” of the Hunger Drive. Beginning with the 
theoretical article of E. E. Anderson (1941a) there has been a continuing 
interest in whether the hunger drive can be conditioned to environmental 
cues or. as Anderson put it “externalized.” 3 There seems little reason 
to doubt that the performance of animals in a particular situation is 
di:7. r.-nt if that m‘« mil ion has be,*n associated with hunger than if it has 

• r. K, Ar. Or* (5M1-, 1ft lb, IPllr. lfsi.l). 
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not To illustrate, Calvin, Bicknell, and Sperling (1953) showed that rats 
which had been placed in a striped box when they were hungry, over 
a period of 24 days, subsequently ate slightly more there than a group 
which had had the same experience when they were not hungry. Such 
evidence, however, only serves to raise the important question : Precisely 
what effect does the association between environmental cues and hunger 
have upon the organism? The answer to this question is not known. It is 
clear that, if conditioning is involved, it is not, strictly speaking, drive 
that is conditioned, inasmuch as only responses are conditionable. Denny 
and Belian (1956) have considered the possibility that demonstrations 
of conditioned hunger drive often reduce to instances of conditioned 
approach behavior. In other cases, learning may not be involved at all. 
Myers and Miller (1954) have called attention to the fact that learning 
thought to be motivated by an acquired hunger drive is sometimes moti- 
vated by an exploratory tendency. And Hall (1956) has shown that the 
animal’s normal increase in activity in the presence of distinctive environ- 
mental stimulation is enhanced by food deprivation. Thus, for a final 
time, a basic question about the functioning of motives comes down to a 
question of the responses produced by the drive-establishing operations. 

2. Social Techniques. The studies of the acquisition of motives re- 
viewed in the previous sections were all ones in which a drive was estab- 
lished by a routine which followed a classical conditioning paradigm. 
That is, in each, the attempt was to provide a neutral stimulus with the 
properties of a drive by pairing it with an existing motive. It may be, 
of course, that the association developed was between the previously 
neutral stimuli and the respo?tses produced by the drive state; but this 
does not alter the nature of the process involved. 

There exists a parallel literature on acquired motivation in which the 
underlying mechanism is more like instrumental learning. Tolman (1942, 
1943) has given us an account which is fairly representative. Figure 13.11 
summarizes his view. The essential idea in this theory is that, in infancy, 
the individual has only a set of biological drives. Inevitably these drives 
are subjected to frustration and new techniques are developed to satisfy 
them, ‘Whatever techniques lead to relief from frustration are learned, 
and by principles which are more typical of Hull’s theory than of Tol- 
man’s, they become characteristic of the individual’s repertory of re- 
sponses to the world. As the drive-conversion diagram also suggests, 
these first primitive adjustments achieved by the individual arc not ade- 
quate to deal with all situations. They, too, arc frustrated with the 
result that new learning occurs and the individual’s reactions to the 
world are modified further. 

It should be noted that, so far, nothing has been said about motives. 

Yet a glance at Figure 13.11 will reveal that several of the social tech- 
niques ore ones which we often describe in motivational terms. Apgres- 




FIGURE 13.11. Tolman’s drive-conversion diagram. The idea is that the social 
techniques develop as a result of the frustration, first, of the biological drives 
and, later, from the frustration of first-order techniques. The specific social 
techniques listed are merely illustrative. The variety of mechanisms acquir- 
able in this way is theoretically very large. (After Tolman, 1942). 
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very many people. An answer to the first question will require a set of 
experiments devised so as to determine whether the social techniques 
possess the properties of other motives. It will be necessary to find means 
of increasing and decreasing the strengths of these motives to determine 
whether their effects upon behavior are appropriate. Is it possible, for 
example, to show that the strength of aggression increases when an 
aggressive individual is deprived of the opportunity to satisfy his motive 
in the same way as hunger increases with food deprivation? Conversely, 
can aggression he satiated in the same way as hunger? Scanty evidence 
indicates that the answer to these questions may he affirmative. Feslibacli 
(1955) aroused the hostility of college students and found that an oppor- 
tunity to express this aggression in a projective test reduced this motive, 
after the manner of consummatory responses in general. Similarly, 
Gewirtz and his associates have shown that deprivation and satiation of 
the need for social companionship have appropriate influences on the 
behavior of children learning for reward in the form of expressions of 
approval (Gewirtz and Baer, 1958a, 1958b; Gewirtz, Baer, and Roth, 
1958). Confirmation of such results will do much to establish the motiva- 
tional status of the social needs. 

The answer to the second question is even more difficult. One form of 
answer was suggested by Woodworth (1918) in liis proposal that habits 
can become drives. Essentially the same answer is to be found in All- 
port’s (1937) conception of functional autonomy, according to which 
habits become ends in themselves, that is, drives. We have already seen, 
however, that such answers are, at best, descriptive and leave the main 
problem still unsolved. 

SUMMARY 

Variations in the intensity of activity in a single individual from time 
to time, as well as from individual to individual, suggest the desirability 
of developing a concept to deal with the energy characteristics of be- 
havior. Motivation is such a concept. 


The General Properties of Motives 
In addition to its capacity to provide behavior with energy, it has 
sometimes been proposed that motivation also guides or directs behavior. 
There is no doubt that motivated behavior is directed, and there is gen- 
eral agreement that the mechanism through which this guidance is ac- 
complished is that of drive stimulation. There is less agreement, however, 
on the relationship between drive and drive stimulation. Some writers 
have maintained that one or the other of these two conceptions is dis- 
pcnsable. We have taken the position that both of them arc of value, but 
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that it is important to distinguish between the guidance and the ener- 
getics of behavior. The term motive should, we think, be reserved for the 
latter function. 


Functions Relating Motives to Behavior 

Since motivation is conceptualized in terms of energy, it is natural to 
expect that increasing the strength of a motive will always produce an 
increase in the vigor of behavior. In the majority of cases, the effect of 
increased motivation is probably in the expected direction ; but there are 
enough exceptions to raise questions. In numerous instances, inex-eased 
motivation has been found to produce first an increase and then a de- 
crease in response strength. In a few others the effect has been entirely 
interfering. 

Careful examination of these functions leads to two important points : 
(1) Basically, the effect of increased drive probably is to increase the 
vigor of behavior. The responses so energized, however, may not be 
those under observation in a particular experiment. In that event, the 
effect of increased drive will vary depending upon the compatibility be- 
tween the energized response and the response being measured. (2) Even 
where the effect is to produce a uniform increase in the magnitude of 
several measures of the strength of a particular drive, the functions may 
be quite different for the different measures. 


The Effect of Drive on Learning 

One of the most important theoretical questions in the field of learning 
is whether the level of learning (as distinguished from performance) 
varies with the level of motivation at the time of learning. Superficially 
considered, experiments designed to answer this question appear to have 
yielded positive and negative answers with about equal freqency. Further 
examination of the evidence, however, suggests another interpretation. 
It may be that experiments which have seemed to demonstrate that drive 
does affect learning are actually irrelevant, because somewhat different 
responses were learned under the different training drives Thus the 

hypothesis that motivation has no effect upon learning is yet to be dis- 
proved. 
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provide distinctive stimuli to ■which the organism can learn to respond, 
and that such learning can occur quite quickly under optimal conditions. 
There is reason to believe that these drive stimuli provide satisfactory 
cues for the guidance of behavior which is unrelated to the drive with 
which they are associated. Moreover, it has been shown that drive stimuli 
provide a dimension along which stimulus generalization occurs. 


Acquired Drives 

At least from the time of Watson, the idea that drive and emotions are 
largely acquired has been very popular in American psychology. Today 
it seems probable that there has been a serious tendency to overestimate 
the importance of learning in relation to motives. But this is not to say 
that motives are never acquired or uninfluenced by learning, because 
they obviously are. In the study of acquired motivation, it is possible to 
identify four somewhat separate lines of approach to the problem : 

1. The Modification of Instinct. There is no doubt that the biological 
drives such as hunger, thirst, and sex depend upon experience, at least 
for their mode of expression. There is also a very small amount of in- 
formal evidence that instinctive reactions are conditionable. 

2. Incentive Motivation. In previous chapters (p. 170) the point 
has been made that rewards produce motivational effects. Operating in this 
way rewards are called incentives. The mechanism underlying the de- 
velopment of incentive motivation seems to be a matter of conditioning 
goal responses to previously neutral aspects of the learning situation. 

3. Fear as an Acquirable Drive. By far the largest amount of experi- 
mental work has been done on conditioned fears, which are capable of 
motivating behavior. Beginning with the classical study of Watson and 
Rayner, there have been numerous investigations to show (a) that condi- 
tioned fears display the various characteristics of other classically condi- 
tioned responses, ( b ) that fear will serve as a motive for learning, and 
(c) that fear-reduction is a reinforcer. There is some evidence that the 
arousal of positive motives in a particular situation also influences be- 
havior in that same situation on later occasions. There is still a question, 
however, as to whether this effect is properly considered motivational. 

4. Social Techniques. Finally, it is quite obvious that such tendencies 
in human behavior as those toward imitation, aggression, achievement, 
and loyalty are learned. And, in common speech, they are often spoken 
of ns motives. Whether they qualify psychologically ns motives remains 
a question. They should, we think, be regarded ns motives if they show 
the same properties ns other motives; for example, if they increase in 
strength with deprivation and are satiated with consummately behavior. 
There is fragmentary evidence that they do display these characteristics. 
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Applications to Personality 


Although the laws of conditioning are important in their own riglit, 
as statements of tlie nature of learning in specially devised experimental 
situations, most psychologists probably feel that these laws would be 
much more important if it could be demonstrated that they also apply 
to complex behavior nearer to everyday human activity. Accordingly, 
there have been numerous^ attempts to apply the principles of simple 
■ learning to such behavior. "The most recent efforts in this direction have 
extended the principles of conditioning and simple learning to rote 
memorization (Hull, Hovland, Ross, Hall, Perldns, and Pitch, 1940), the 
acquisition of motor skill (Ammons, 1947; Kimble, 1949), social be- 
havior (Miller and Dollard, 1941), and personality and psychotherapy 
(Dollard and Miller, I960; Shoben, 1949). Attempts such as these raise 
basic questions about the relationship between conditioning and other 
forms of behavior. On the one hand, some writers have argued that all 
behavior must be an expression of a single set of laws which apply at all 
levels of complexity, from the conditioned response in lower animals to 
the most elaborate mental functioning of man. To others it has seemed 
that new principles will be required to deal with complex behavior and 
that no combination of the principles of conditioning could ever serve to 
explain the complete range of human activity. At the present time it 


would be a mistake to accept either of these positions. It may, some day, 
be known whether the laws of conditioning do or do not explain (say) 
psyehopathological behavior. But that day is still far in the future. For 
the time being all that is possible is to attempt the explanation of com- 


plex phenomena in simpler terms. It is to be expected that the resulting 
explanations will be incomplete and imperfect. Complex behavior, if it 
is explainable at all in these terms, certainly involves the simultaneous 
operation of many principles of conditioning. Unfortunately these prin- 
ciples are not exactly known, and we know even less about the way in 
v.hich th* y combine and function together. These facts, together with the 
r-mideration that knowledge in the field of personality is'' also in a very 
primitive stam • of development, mean that there will he many points a*t 
v.um learning theory can olTor only the grossest of speculations in deal- 


•43G 



APPLICATIONS TO PERSONALITY 437 

ing with problems of personality. And there will be other cases in which 
conditioning principles suggest ideas which do not appear in personality 
theory. Such disagreements and lacunae define problems for fature re- 
search in both areas. 

GENERAL NATURE OF THE LEARNING THEORY 
APPROACH 

One of the salient features of the learning theorists’ approach to the 
topic of personality has been a tendency to try to co-ordinate psychiatric 
(most often psychoanalytic) concepts to the familiar concepts in the field 
of learning. Thus personality has been defined in terms of habit (Guthrie, » 
1938) and such processes as generalisation, discrimination, reinforce- 
ment, extinction, and partial reinforcement have been advanced in an 
effort to explain the adjustment mechanisms familiar to the psychiatrist 
by means of concepts familiar to the learning theorist. Experimental 
analogues of psychoanalytic phenomena have been demonstrated, often 
in studies with lower animals. These studies have included efforts to pro- 
duce concrete instances of the processes regarded as important for 
therapy. 

The Influence of Psychoanalysis 

At first blush the fact that the learning theorists have drawn quite 
heavily from Freudian theory seems strange, because of what appear to 
be vast differences in approach to psychological problems. Although at- 
tempts have been made to represent Freud as an experimentalist, they 
have not been convincing ;‘Freud’s great contribution is his penetrating . 
insights into the dynamics of human behavior. The learning theorists, on 
the other hand, have always remained close to the laboratory and have 
insisted on more objective evidence than Freud usually had for his 
interpretations. 

A closer loolc at Freud’s psychology, however, reveals a number of 
strong similarities which the learning theorist finds appealing. The most 
general of these is the commonly shared assumption that behavior is 
determined in part by the events in an organism’s past. Even more im- 
portantly, the concepts of both systems are historical in nature. Although 
other systematists, notably Lewin (1935) and liis followers (for example, 

J. F. Brown, 193G) , admit that behavior is, as n matter of fact, historically * 
determined, such theorists have developed ahistorical theories and deny 
the usefulness of attempting to understand contemporary behavior in- 
terms of past causes. Thus, the learning theorist has been drawn to psy- 
choanalytic thinking by a basic commitment implicit in the nature of hi* 
subject matter. 
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Beyond this large and pervasive parallel between Freudian psychology 
and learning, a parallel involving matters of general strategy, there are 
numerous other more limited similarities which give this theory appeal 
for the learning theorist : 

1. Freud’s approach to motivation is a kind of variation on the con- 
cept of homeostasis which has been implicit in the thinking of many 
learning theorists. In the field of physiology, homeostasis refers to the 
regulative tendencies of the body by which certain constancies are main- 
tained. The mechanisms for the control of body temperature, salt content 
of the body fluids, and water balance in the cells are typical examples. 
"When this concept is applied to motivation, drives become tendencies to 
correct imbalances produced by lacks or excesses in the organism’s en- 
vironment. Freud’s treatment of motives took this form and the adjustive 
behavior of the organism was regarded as conservative in this homeostatic 
sense. 

2. As a similarity closely related to the point above, Freud’s concept of 
the pleasure principle obviously paraphrases the law-of-effeet. The ener- 
gies of the id, in Freud’s theory, are conceived as throwing the individual 
into a state of painful tension: Adjustive behavior is such as to relieve this 
tension. In Freudian theory both normal and abnormal behavior serve 
the same end, tension reduction. This is obviously very similar to the 
principle advocated by Hull in his drive-reduction theory of reinforce- 
ment. In what appears to be a contrast to Hull, Freud held that the ideal 
state of the organism is not the complete absence of tension, but rather 
one of tension at a mild and optimal level. 

3. Certain aspects of Freud’s description of the developing personality 
seem easily translatable into the concepts of classical and instrumental 
conditioning. The psychological development of the individual involves 
two mechanisms, which Freud called primary and secondary processes. 
Briefly the essentials of these processes are as follows : 

a. Primary process. The infant’s first reactions to the world are gov- 
erned by the pleasure principle and are motivated by selfish needs for 
immediate bodily satisfaction. "When satisfaction occurs, the objects which 
produce it become associated with tension reduction. As a result, the in- 
fant. when frustrated, is able to call up an image of the conditions (food, 
fondlimr. rocking, dryness) which will relieve its tension. Up to this point, 
tho concept of primary process seems to be little more than classical con- 
ditionin'.: described in terms somewhat similar to those employed by 
theorists who accept the idea of perceptual learning. Along the same lines 
Ftvud hold that, because of its immaturity, the infant is unable to d£ 
t.ntruish between object and image, with the result that the mere thou-ht 
in action is somewhat tension-reducing. This extension of the con- 
e,p; —ms to be almost exactly the same thing as was involved in the 
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extension of classical conditioning principles to cover the phenomenon of 
secondary reinforcement (p. 170). 

b. Secondary process. The primary process provides a poor basis for ad- 
justment to the environment. The demand for immediate satisfaction and 
the inability to distinguish wish from object are clearly maladaptive in 
many situations. With his increasing intellectual powers and better con- 
trol over the environment, the infant develops in directions which tend 
to correct this state of affairs. This development is the secondary process, 
as a result of which the infant comes to distinguish between self and 
environment, learns techniques for obtaining desired goals, and is able 
to forego immediate pleasure. ‘Jle reacts more and more in terms of the / 
reality principle rather than the pleasure principle. The secondary process f 
is quite plainly analogous to instrumental learning. Indeed the picture of 
instrumental behavior sketched by the psychoanalysts is not much dif- 
ferent from that of the S-R reinforcement theorists. Through the primary 
process the organism is capable of knowing what will satisfy its needs. 
The secondary process provides the means through which satisfaction be- 
comes a reality (Hall, 1954, p. 29). 

4. Elements of the personality dynamics proposed by Freud bear a 
resemblance to the conflict analysis developed by N. E. Miller. In psycho- 
analytic terms, there is continual friction between the demands of the id, 
which are primitive, asocial, biological motives, and the superego which 
contains the mores and taboos of one’s society. This is essentially on in- 
ternalized approach-avoidance conflict in which an individual’s tendencies 
toward sex and aggression clash with tendencies to feel guilty about 
exactly these impulses. 

5. Anxiety is important in Freud’s thinking (Freud, 1926) in many of 
the ways in which learning theory tends to use the concept. Anxiety pro- 
vides a motivational basis for adjustment since it is experienced as un- 
pleasant. And relief from anxiety is regarded as an important goal. More- 
over, anxiety was thought of as providing the cues (drive stimuli) which 
evoke the various mechanisms of adjustment. 

6. Certain of the basic phenomena of learning are represented, at least 
approximately, in psychoanalytic thought. Generalization, for example, 
is clearly a part of what is referred to in predicate thinking, a process 
involved in symbol development. A knife becomes a symbol for tlic male 
organ because both are long and pointed. Habit appears in the Freudian 
conception of repetition compulsion. The individual’s behavior tends to 
be consistent, and his various adjustments to the world tend to take 
similar forms from one occasion to another, even though the details may 
change: “An adult may change his job or his wife or his hobbies quite 
frequently, but the new job or wife or hobby bears n close resemblance 
to the old one” (Hall, 1951, p. 117). Acquired drives are represented 
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mueli as Tolman (1943) does as instinct derivatives or new object 
cathexes. The id cannot always achieve its aims directly and must make 
compromises, which take the form of displacements or sublimations of 
the primitive urge. Attempts to satisfy sexual tensions by physical 
activity or verbal expressions of aggression are examples. Because such 
indirect expressions do partially relieve tension they become a part of the 
individual’s characteristic adjustment and constitute the basis for the 
development of very persistent motive states. 

It thus becomes clear that Freudian psychology contains much that is 
directly translatable into the concepts of learning. Partly for this reason 
the learning theorist interested in personality has tended to develop 
theories and experimental ideas closely in touch with psychoanalytic 
thought. There are however certain historical and methodological points 
which need to be made: (1) The interpretation of psychoanalysis in 
terms of the psychology of learning has not occurred as a grand hypo- 
thetico-deductive tour de force. What has so far been accomplished has 
been done in a piecemeal and fragmentary way, with much looking to the 
writings of Freud after the experimentally verified facts are in. The 
process is still going on. (2) Both psychoanalysts and psychologists recog- 
nize that learning theory has managed to deal only with the most super- 
ficial aspects of personality as interpreted from the analytic point of 
view (Kris, 1951). This is largely because the psychology of learning, at 
the present time, has much difficulty in dealing with symbolically medi- 
ated behavior, although progress in this direction has been made (Dollard 
and Miller, 1950). 


EXPERIMENTAL NEUROSIS 

Still another parallel between the learning theorist’s approach and that 
of Freud has been a greater interest in neurotic than in psychotic forms 
of maladjustment. One of the earliest laboratory attacks on problems of 
personality consisted in the study of experimental neuroses, a term which 
has often been criticized but continues to be used (Liddell, 1956). This 
field of application for conditioning procedures and theories began with 
some experimental observations by Shenger-Krestovnikova in Pavlov’s 
laboratory in 1914 (Pavlov, 1927, p. 291ft.): A dog was trained to salivate 
when a luminous circle was projected on a screen. After the conditioned 
response was well established, a discrimination was obtained between the 
crele and an ellipse with a ratio between the semi-axes of 2-1 The dis- 
crimination was acquired comparatively quickly. The shape of the ellipse 
T n T- d by Sta " PS ’ until U was alm °st that of a circle. Durin- 

' 1 7 r r h, " ,i<m lining continued , the circle was 
' -force, net. but 11, c ellipse was not. Finally an ellipse 

“ n ““ '” mU ' a: ' cs ot 8 :S reached. At this point discrimiL 
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tion not only did not improve, but throughout three weeks of training it 
became worse. 

At the same time the whole behavior of the animal underwent a marked 
change. The hitherto quiet dog began to squeal in its stand, kept wriggling 
about, tore off with its teeth the apparatus for mechanical stimulation of 
the skin, and bit through the tubes leading from the animal’s room to the 
observer’s, all behavior which had never occurred before. On being taken 
into the experimental room the dog now barked violently, which was also 
contrary to its earlier tranquility. In short, it presented symptoms of a 
condition which, in a human being, we would call neurosis. Among the 
additional symptoms reported by later investigators have been (1) signs 
of anxiety such as whining and trembling, (2) a breakdown of the pre- 
cision of the CR, with the latency of the response becoming extremely 
variable, (3) refusal of the dog to eat in the experimental apparatus or 
room, (Dworkin, 1939), and (4) signs of strong “inhibition” such as 
yawning drowsiness and sleep (Muncie and Gantt, 1938). There are three 
circumstances under which the neuroses are said to arise : the application 
of a strong conditioned stimulus after the animal has become accustomed 
to a weak one (overstrained excitatory processes ) ; protracted inhibition, 
as in excessive delays (overstrained inhibitory processes) ; and clash of 
excitation and inhibition, as in an overly difficult discrimination. 

Experimental Neuroses in the Sheep 

Liddell and his co-workers at Cornell have studied conditioned re- 
sponses in sheep for many years and have observed experimental neuroses 
since 1926. Generally speaking, there is good agreement between their 
observations and those of Pavlov. Experimental neuroses have been pro- 
duced in a number of ways which include: (1) requiring a difficult dis- 
crimination (Pavlov’s clash of excitation and inhibition), (2) employing 
a long series of extinction trials or too long a delay between CS and UCS 
(Pavlov’s overstrained inhibitory process), and (3) carrying out n long 
series of alternating reinforced and non-reinforced trials at a regular 
seven-minute intertrial interval. Among the symptoms of neurotic be- 
havior are the following : (1) hyperirritability, (2) resistance to entering 
the laboratory, (3) motor abnormalities including persistent tic-like 
movements and tremors, (4) changes in social behavior, including refusals 
to join the flock and symptoms of “suspicion” and “aggressiveness,” 

(5) loss of ability to withhold responses to negative conditioned stimuli. 

(G) loss of the ability to delay responses to positive stimuli, (7) autonomic 
symptoms including nbnormaltics of respiration, eliminative function , 
and heart action, and (8) abnormal effects of drugs (Liddell, 193S; 
Anderson and Parmcntcr, 1941). One of the striking facts reported by 
the group at Cornell is that experimental neuroses are very persistent, 
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lasting for 13 years or more (Liddell, 1956). On the other hand, Liddell 
(1956) also reports that experimental neurosis in the sheep is frequent y 
so incapacitating that the experimental animals die early. In particular, 
they become easy prey for marauding dogs. 


Experimental Neurosis in the Rat 

Successful development of experimental neurosis in the rat has been 
reported by Cook (1939). Prom among a group of six rats, three devel- 
oped disorders of behavior in a situation described as follows : 

The rats were strapped to a stand so that the only sizeable limb movement 
possible was a flexion of the right foreleg. Under certain conditions such a flexion 
was rewarded with a food pellet; under other conditions it was punished with an 
electric shock. Observations indicated that the animals experienced two principal 
stresses: the first, when they were required to delay the food-bringing flexion 
until they received a bright-light stimulus; the second, when they were required 
to make a very difficult discrimination between a bright-light stimulus which per- 
mitted a food-bringing flexion and a dim-light stimulus which prohibited such a 
flexion on pain of shock. Because, under such conditions, the organism was re- 
ceiving simultaneously stimuli to the excitation and inhibition of the same re- 
sponse, the stresses experienced have been characterized as comprising a “clash” 
between the neural activity of initiating a response and the neural activity of 
inhibiting the same response. (Cook, 1939, pp. 307-308) 

Cook points out that restraint of activity is an essential feature of the 
experimental situation leading to abnormal behavior. In three other 
conflict problems, which permitted more freedom in bodily activity, no 
disturbance was produced. 


Experimental Neurosis in the Cat 

Studies of experimental neurosis in cats have been reported by Masser- 
man and his associates (for example, Masserman, 1943). In these experi- 
ments cats were trained to operate a switch to produce a signal that food 
was available and then to get food delivered into a receptacle by an auto- 
matic delivery mechanism. Then, at the moment of feeding, an unpleasant 
axrhlast was directed across the animal’s face or a shock was delivered to 
the f c-t . The conflict thus produced led to a variety of behavioral changes 
mcUnhnc d) re.t lowness and agitation, or, sometimes, passivity, (2) fear 
responds to the feeding signals such as trembling, rapid and irregular 

r " spir f ,,on - crouchin P' h Wins, and attempting to escape from 
t.m apparatus {.;) refusal to cat. (4) “compulsively” stereotyped be- 

T behavior such as preening, “immature” 
... ^r- ’.Mu or Mtiemsh playfulness. 
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Evaluation of Research on Experimental Neurosis 

A question which occasionally arises in connection with the experi- 
ments just described is whether they have produced a real neurosis. With- 
out further specification of the meaning of the term neurosis this question 
has no answer. If the implication in such questions is that there exists 
some disease entity, neurosis, which, like poliomyelitis, people get and 
lower animals can also get, the answer must be a rejection of this idea. 
On the other hand, the behavior which we call neurotic has certain char- 
acteristics which seem to be represented in the studies of experimental 
neurosis: (1) Neurotic behavior is the product of stress and conflict. It 
will be observed that in most of the demonstrations cited above a conflict 
is plainly evident between reinforced and non-reinforced, or punished, 
action tendencies. (2) Neurotic behavior is characterized by anxiety. This 
implies punishment of some sort; and, again, it will be noted that a 
noxious stimulus has usually figured in the studies of experimental neu- 
rosis. (3) Neurotic behavior occurs, in part, because of conflict which is 
inescapable. In the animal experiments, physical restraint has always 
been a part of the procedure, and one of the typical results has been a 
resistance of the subjects against entering the situation where conflict 
occurs. t (4) Neurotic behavior is marked by a set of symptoms that are 
frequently very unusual by social standards and always only partially a 
solution to the conflict. Analogous behavior has often been observed in 
the studies of experimental neurosis. On all of these counts, the term 
experimental neurosis seems justified and not unduly anthropomorphic. 
But, in one respect, the studies of experimental neurosis do deserve sharp 
criticism: the procedures employed have typically been haphazard, the 
number of subjects run under strictly comparable conditions small, and 
the behavioral measures at best unquantified and often grossly subjective. 
In addition to this, the phenomenon itself is exceedingly complex. The 
characteristics of neurotic behavior just listed suggest some more par- 
ticular lines of investigation with relevance to personaliity. These involve 
the topics of anxiety, conflict, and symptom formation. A short section 
will be devoted to each. 

ANXIETY 

In applications of learning theory to problems of personality the con- 
cept of anxiety occupies a central position. The general point of view 
parallels that of psychoanalysis in assuming that anxiety plays a double 
role, being on the one hand a drive, and on the other, a source of reinforce- 
ment through its reduction. Studies cited earlier show that this is a 
reasonable assumption. Anxiety is capable of producing restless activity, 
and dements of such activity leading to anxiety reduction arc learned. 
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Types o£ Anxiety 

Drawing Ms ideas from laboratory experimentation almost exclusively, 
tlie learning theorist attributes anxiety to a conditioning process m 
which neutral stimuli have come to stand for punishment, pain, and fear. 
The nature of the evidence for this conception leads to a limited view ot 


anxiety for two reasons. 

1. The noxious unconditioned stimuli in most experiments have pro 
duced physical pain, which means that it is dangerous to generalize to 
fears based upon other noxious stimuli. It seems clear, however, that 
many other forms of stimulation are innately fear producing. Some of the 
stimuli which have been shown to be unconditioned fear stimuli are : (a) 
in birds of certain species, the shadow of a hawk (Tinbergen, 1951) ; (5) 
in monkeys, the decapitated head of another monkey (Hebb, 1946) and 
even certain toys (Harlow, 1958) ; (c) in cats, air blasts in the face 
(Masserman, 1943) ; (d) in the human infant, sudden loud noises and 
loss of support (Watson, 1925) ; and (e) in rats, being placed in a large 
open field. These stimuli are, of course, in addition to painful stimuli, 
which appear to be negative stimuli for all species. Hebb (1946) classifies 
innate fears into three categories : (a) fears due to conflict which include 
fears induced by pain, sudden loud noises, dead or mutilated bodies, and 
strange persons or animals, (5) fears due to sensory deficit, which include 
fears based on loss of support, darkness, and solitude, and (c) fear pro- 
duced by physiological disturbances. Examples of the latter are provided 
by the psychotic fears frequently associated with pellagra. It may be that 
endocrine disturbances also sensitize fearful reactions which individuals 
without the disturbance do not display. Beyond this, it is quite clear that 
certain acquired fears also can serve as the motivational basis for much 


ad justive behavior. Phobias and fear of loss of love are two examples. 

2. Conceptions of anxiety based exclusively upon laboratory research 
tend to overemphasize the importance of the fear of objective situations. 
The psychoanalytic literature contains evidence that anxiety occurs in 
many different forms. Freud distinguished three varieties of anxiety, 
objective, neurotic, and moral. Objective anxiety is anxiety about events 
in the real world and includes anxieties about physical harm and eco- 


nomic privation. The cues provided by these forms of anxiety ordinarily 
lead the individual to take measures to avoid the threatening danger. He 
looks for a hiding place, a job, or a means of defense, depending upon the 
nature of the threat. Neurotic anxiety (which is not the exclusive prop- 
erty of neurotic people) is an anxiety prompted by desires of the id • it is 
a fear of expressing antisocial, sexual, aggressive urges in the form of 
•-rune impulsive act. Its symptoms, according to Freudian psychology are 
. found in free-floating anxiety, phobias, and panic. Moral anxiety experi- 
as emit, shame, or a pang of conscience, comes from the superego. 
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In the past certain sorts of behavior -were punished by the parents. Insti- 
gation to similar behavior now calls up the anxiety thus established. This 
is moral anxiety. 

In terms of practical adjustment, it is clear that moral and neurotic 
anxiety will be harder to deal with than objective anxiety. It is usually 
possible to escape from objective anxiety by running away, but in the case 
of the other two it is not. In any environment, performing the guilty act, 
almost performing the guilty act, or even thinking about it, are able to 
elicit unpleasant anxiety. There is a tendency among the psychoanalysts 
to distinguish more sharply between objective anxiety and the other two 
than between moral and neurotic anxiety. In psychoanalytic thinking, 
these latter two are intrapsychic. In the concepts of the learning theorist, 
they are conditioned to aspects of the individual’s behavior, whereas ob- 
jective anxiety is occasioned by stimuli in the external world. In this case 
the concepts of the learning theorist may provide the means of making a 
distinction where Freudian psychology finds it difficult. From the descrip- 
tions of moral and neurotic anxiety, it sounds very much as if the differ- 
ence is in terms of the stimuli to which the anxiety is conditioned. If 
neurotic anxiety is a fear of expressing antisocial urges , the suggestion is 
that, in this case, the anxiety-evoking stimuli are drive stimuli. A fear of 
performing certain acts, as is said to be the case in moral anxiety, on the 
other hand, may be elicited by response-produced stimuli. 

The Origin of Anxiety 

Anxiety is an inevitable by-product of socialization. In infancy the 
child has no way of knowing what behavior society rewards and punishes; 
it responds in terms of its selfish needs. Some of these are socially taboo 
and are punished by the parents. Quite obviously, from the point of view 
of the child, the adjustive thing to do is to inhibit the unacceptable be- 
havior and, thereby, avoid punishment. Inevitably this is the child’s 
solution to the problem. The solution, however, is, at best, imperfect for 
several reasons: (1) It leaves the original drive unsatisfied. It must be 
put aside or even forgotten. This is a point which the psychoanalysts 
stress, in their view that acute anxiety is always symptomatic of repres- 
sion. (2) The reinforced and punished acts to be discriminated create real 
problems. The child must distinguish between rewarded forthrightness 
and punished aggression, between proper respect and sissified timidity, 
between craven cowardice and graceful concession, between brotherly 
affection and incestuous lust, between the joys of male companionship 
and homosexuality, between standing up for one’s rights and selfish 
hostility. Probably no one learns to recognize all the necessary nuances 
in Ins own behavior, or to achieve a satisfactory correlation of the ones he 
does discriminate to the requirements of the environment, (3) The origin 
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of much anxiety is to be found in incidents which occurred before the 
child had learned to talk. Punishment for soiling during toilet training, 
for infantile sexuality, for lapses of table manners, and for early aggres- 
sion all instill anxiety for behavior which bears no verbal label. In later 
life such verbal labels are of great assistance in developing discrimina- 
tions between tabooed and nontabooed acts. The labels are usually easier 
to discriminate than the stimuli arising from the acts themselves. In early 
childhood, however, anxiety may be conditioned to vague, poorly differ- 
entiated aspects of behavior. (4) In a sense it is impossible for the child 
to make completely accurate discriminations and to develop responses 
perfectly calculated to avoid all punishment. Too much depends upon the 
inconsistent moods and insensitive perceptions of parents and others in 
control of the child’s life. Sometimes certain acts (for example, boisterous 
play) are encouraged; sometimes they are punished. Beyond this, there 
is no guarantee that such consistency as parents show will be matched by 
a similar predictability in the behavior of nurses, babysitters, grand- 
parents, or teachers. 


Avoidable vs. Unavoidable Punisbment and tbe 
Strength of Anxiety 

It is a common-sense opinion that unavoidable punishments, where the 
individual has no control over the situation, lead to greater fear than 
avoidable punishment. There is also experimental evidence to support 
this idea. In avoidance conditioning experiments, the subjects frequently 
show decreased general anxiety as they learn the response which avoids 
punishment. Particularly in experiments using the wheel-turning ap- 
paratus, there tends to be a great deal of instrumental activity between 
trials. Tbe animal “rehearses” the avoidant response of wheel turning 
with great persistence, although the danger signal is absent. With prac- 
tice, however, this behavior disappears almost completely. It is as if the 
original high level of between-trial activity arose from a “feeling of 
helplessness” and that learning the avoidant response created a “sense 
of mastery” with a resulting diminution of fear of the general situation 
(Coppock and Mowrer, 1947; Mowrer and Lamoreaux, 1951). 

In a different approach to the relationship between fear and the 
avoidability of punishment, Mowrer and Yiek (1948) employed, as a 
measure, inhibition of eating. Their procedure consisted of offering the 
rat food on the end of a stick and, 10 seconds later, whether the rat ate or 
not, turning on a shock. The animals in one group had control over the 
duration of the shock because it was terminated as soon as the rat jumped 
off the electrified grid. Another group received exactly the same treat- 
ment, except that the animals in this group had no control over the shock. 
Each subject in this latter group was paired with one in the former and, 
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on each trial, given the same amount of shock received by its partner. 
The difference in behavior in the two groups of animals was very striking. 
The 10 subjects in the first group with control over shock duration failed 
to eat on an average of only 1.6 of 14 trials on which meaningful com- 
parisons can be made. The animals in the group with no control over shock 
duration failed to eat on 8.5 trials. It thus appears that the unavoidable 
shock increased the strength of fear. In other studies the same effect may 
occur because of a difficulty in recognizing the significant environmental 
cues. This would also make punishment unavoidable. 


Effects of Anxiety on Behavior 

Anxiety is commonly recognized as an unpleasant experience and as a 
disrupter of behavior. The learning theorist takes it as his task to explain 
these familiar influences within his own conceptual framework. The fol- 
lowing two ideas have dominated research in this area: (1) Anxiety is 
conceived as a drive, with anxiety reduction playing the role of rein- 
forcer ; and (2) as with other drives, anxiety has distinctive stimuli asso- 
ciated with it. These stimuli serve as cues to which responses can he con- 
ditioned, with the result that the individual whose anxiety is aroused may 
be expected to show a characteristic set of responses. Although individuals 
who favor one or the other of these views sometimes have argued as to 
which is correct, it should be recognized that, as with other drives, anxiety 
almost certainly functions both as a motive and as a producer of 
stimulation. 

Most of the research on anxiety carried out by psychologists interested 
in learning has been done to test the implications of the first of these 
propositions, the idea that anxiety is a drive. The line of reasoning behind 
such studies was developed in some detail in the previous chapter (p. 
408). In very brief review, the assumption of a multiplication relation- 
ship between drive and habit strength leads to the following predictions: 
(1) In simple, uncomplicated learning situations, such as classical condi- 
tioning, where only one response is elicited, anxiety should facilitate 
performance. This is because anxiety, as an irrelevant drive, contributes 
to the total motivational level of the subject and increases the quantity 
obtained when habit strength and drive arc multiplied. (2) In com- 
plex, even two-choice situations, the effect of increased drive depends 
upon two further considerations: (a) the nature of the dominant habit 
evoked in the learning situation, and (6) whether the response which 
corresponds to this habit is right or wrong. If it is right, increased moti- 
vation (high anxiety) should facilitate performance, just as in the ea«e 
of noncompctitional learning. If it is wrong, the situation becomes more 
complicated. Initially anxiety should interfere with performance, sinee 
it energizes incorrect responses. As the correct responses arc mastered, 
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however, there should he a point at which these latter responses become 
dominant; and from then on, anxiety again may be expected to facilitate. 
In complex learning situations, thus, the effects of anxiety upon behavior 
may be facilitating, interfering, or first interfering and then facilitating, 
depending upon the correctness of the original response tendencies of 
the subject and the manner in which these tendencies change with 
practice. 

Tests of these deductions about the effect of anxiety on learning have 
most often made use of Taylor’s (1953) Manifest Anxiety Scale (MAS). 
The MAS consists of a selection of items from an objective test, the Min- 
nesota Multiphasic Personality Inventory, judged by clinicians to be 
diagnostic of anxiety. In the main, the items are one-sentence descrip- 
tions of anxiety symptoms, and the subject is asked to indicate whether 
each is characteristic of him. The measure of anxiety is the number of 
such symptoms to which the individual admits. In typical experiments, 
subjects with scores indicating very low and very high anxiety are com- 
pared in their performances on some learning task. 

The original studies of the effect of anxiety upon learning (Spence and 
Taylor, 1951; Taylor, 1951) employed eyelid conditioning as the learn- 
ing procedure. The results of one of these appears in Figure 14.1. The 
predicted difference in favor of high-anxiety subjects occurs and its 
magnitude increases with practice. This result has been obtained in a 
number of studies (Spence, Farber, and E. Taylor, 1954; Spence and 
Ross, 1957). Studies involving tasks which are more complex than eyelid 
conditioning have employed, to mention a variety, verbal maze learning 
(Taylor and Spence, 1952), stylus maze learning (Farber and Spence, 

1953) , serial rote learning (Montague, 1953), paired-associates learning 
(Ramond, 1953), discriminative eyelid conditioning (Hilgard, L. V. 
Jones, and Kaplan, 1951 ; Spence and Beeeroft, 1954 ; Spence and Farber, 

1954) , problem solving (Maltzman, Fox, and Morrissett, 1953), and 
concept formation (Wesley, 1953). The support for a drive theory of 
anxiety from these studies, although generally positive, has been less 
impressive than that obtained in the studies of eyelid conditioning. In 
certain notable cases (for example, Montague, 1953 ; Taylor and Chap- 
man, 1955) high-anxiety subjects have performed better on noncompeti- 
iional tasks and poorer on competitional ones, as it was predicted they 
would. But a more typical finding is simply that high-anxiety subjects 
are inferior on complex tasks, whatever the strengths of correct and 
incorrect tendencies. This suggests that complex learning situations 
evoke interfering responses in highly anxious subjects, and this tends to 
override whatever beneficial effects anxiety may have upon learning in 
situations of this sort. 

Much of the discussion in the literature on anxiety has stressed the 
fact that the influence of anxiety on behavior is exceedingly complex, 
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TRIALS 

FIGURE 14.1. Conditioning curves /or high- and low-anxiety subjects (Toy] or, 
1951). 

an emphasis •which seems wholly justified on the basis of research. 
It has been found that the effect of anxiety depends upon individual 
difference variables such as sex (L ’Abate, 1956) and the intelligence 
of the subjects (Spiclberger and Katzenmeyer, 1959). It has also been 
suggested that subjects with different scores on the MAS probably 
differ in other ways, too, and that these differences complicate the 
interpretation of results. The importance of this fact depends upon 
whether the main interest is in the role of drive (anxiety) in learning, 
as it lias been in the most important research carried out with the MAS, 
or whether the main interest is in the intrinsic nature of anxiety. In the 
latter ease the need for a purer measure is obviously imperative. 


Wlint the Anxiety Scales Measure 
Studies of the relationship between MAS scores and clinicians’ judg- 
ments of anxiety have yielded correlation coefficients ranging from JC 
to ,G8 (Taylor, 19oG) with a median in the neighborhood of .40. This 
suggests that wlmt the MAS measures mnj' be only partly anxiety. Other 
possibilities •which have been suggested are general neuroticism (Franks, 
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1956), introversion (Eysenck, 1955), psycliastlienia and absence of hys- 
teria ? (Eriksen, 1951; Eysenck, 1955), and emotional responsivity 
(Spence, 1958). Tbe one stndy which is most directly pertinent to the 
view of anxiety as an emotional response is that of Runqnist and Ross 
(1959). These investigators obtained a low, but significant, correlation 
of .22 between physiological measures of emotionality, heart rate and 
GSR changes, and MAS scores. They also compared eyelid conditioning 
performances of two extreme groups selected in terms of the physiological 
indicators. Their results indicated that subjects who were highly re- 
sponsive in physiological terms showed superior conditioning. 

One of the questions sometimes raised about anxiety, however meas- 
ured, concerns its relationship to environmental stimulation. The early 
studies, emphasizing the concept of anxiety as a drive, conveyed the 
impression (intended or otherwise) that anxiety, like hunger, was some- 
thing which the organism carried around with him to energize any and 
all habits. This might be called a “chronic” interpretation of anxiety. 
An alternative view, which held that anxiety occurred only in response 
to threatening situations, might be called an “acute” interpretation. 
"With Spence’s recent interpretation of anxiety as an emotional response 
(Spence, 1956, 1958), the “acute” view seems to have become the ac- 
cepted one. Since it is a response, it presumably occurs as a reaction to 
stimulation, and the conventional laws of the evocation of responses 
should apply. Among these would be the proposition that anxiety, since 
it was acquired through punishment, would appear most markedly in 
situations which symbolize or portend punishment. Experimental support 
for such an interpretation comes from the facts that: (X) Eyelid condi- 
tioning, where the MAS has had its most conspicuous successes in pre- 
dicting the outcome of experiments, involves a noxious stimulus. (2) 
High- and low-anxiety subjects apparently perform equally well in 
classical conditioning studies where there is no threat (Bindra, Paterson, 
and Strzelecki, 1955). (3) In several studies, stress and failure have 
been found to disrupt the performance of high-anxiety subjects (Gordon 
and Berlyne, 1954; Lucas, 1952). 

Evaluation of the Anxiety as Drive Conception 

The amount of research which has been instigated by the anxiety-as- 
drive conception is enormous. Since the original studies in 1951, the num- 
ber of articles containing the word anxiety in their titles has increased 
steadily. Despite the fact that the outcomes of these experiments have 
not always supported the original theory, they have led to constructive 
reformulations involving the conception of anxiety as an emotional re- 
sponse conditioned to previously neutral stimuli. 

Perl laps the main shortcoming of research on anxiety is that too little 
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attention lias been devoted to the stimuli which normally control anxiety. 
In the most important cases, these stimuli are internal and response- 
produced. The problems implicit in this fact are almost completely un- 
touched by experimentation, although they are in principle solvable. 
Animals are able to discriminate internal drive states including fear and 
anxiety (Amsel and McDonnell, 1951). They also show some capacity to 
discriminate between self-produced and other anxiety signals. Masser- 
man (1943, p. 84) reports that some of the cats in his studies of experi- 
mental neurosis displayed the neurotic behavior only if they themselves 
pressed the key which administered the feeding-airblast signal. If these 
signals were turned on by the experimenter, little or no neurotic behavior 
appeared. An even more fascinating fact is that deliberate training made 
it possible to reverse this relationship. One cat was made neurotic 
(“bodily manifestations of anxiety, self-starvation, exaggerated startle 
reaction . . .”) by a procedure in which the experimenter delivered the 
signals. When the eat was trained to administer the signals itself, no 
signs of disturbance appeared, although it continued to cower at all 
signals given by the experimenter. Further research along these lines 
could contribute much to our understanding of response-produced 
anxiety. 

CONFLICT 

Conflict occurs when two or more incompatible reaction tendencies are 
instigated 1 simultaneously. This means that conflict is characteristic of 
all choice situations, and that it can be understood in terms of the 
principles which apply to any case where competing habits are aroused. 
Two such cases involving the conflict between short and long paths to a 
goal (p. 142) and between alternative reinforced responses in a discrim- 
ination situation (p. 373) have been discussed previously. 


Types of Conflict 

The classical literature recognizes four types of conflict! approach- 
approach, avoidance-avoidance, approach-avoidance, and multiple ap- 
proach-avoidance. It is obvious at once that these categories derive from 
the two basic action tendencies discussed earlier in connection with 
instrumental learning and reinforcement : approach tendencies based on 
positive reinforcement, and avoidance tendencies based on negative rein- 
forcement. Thus, approach-approach conflict occurs when two approach 
tendencies occur at the same time. Sometimes these two tendencies »n- 

> Tbo term instigation. In its various form*, I* a very osefnl one In that It provides 
a means of referring to the combined effect of drive and stimulation by a single word. 
An net is instigated If the stimuli and motivation for it are simoftaneouity present. 
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volve different goals, and selecting one means giving np the other. This is 
divergent approach-approach conflict. In other situations the two lines of 
action are directed at the same goal, as in the case where the rat chooses 
between long and short paths to food. This is a version of convergent 
approach-approach conflict. 

Avoidance-avoidance conflict occurs in situations where one of two 
undesirable lines of action is required. It may be necessary, for example, 
to do one of two undesirable tasks but not both. 

Approach-avoidance conflict occurs when the same object or situation 
is both attractive and repelling. A child wants to pet a strange dog but 
he has been warned not to. The pigeons in the park seem attracted by 
peanuts held out to them, but unable to take them from the hand because 
of what appears to be fear. It is characteristic of these conflicts that they 
involve fear or anxiety as one of the response tendencies. 

Multiple approach-avoidance conflict occurs in situations where im- 
portant choices must be made ; for example, between careers, job offers, 
roommates, colleges, new homes, or scientific theories. In these cases, 
neither alternative is wholly attractive or unattractive; each has its 
desirable and undesirable features. 

Experimental Analysis of Conflict 

Although there has been some experimental work on approach-approach 
and avoidance-avoidance conflicts (Hovland and Sears, 1938) greatest 
interest has been in the approach-avoidance variety. N. E. Miller and 
his colleagues have provided a series of theoretical and experimental 
studies of approach-avoidance conflict in which basic learning concepts 
are made to play a crucial explanatory role. The experimental studies 
with the laboratory rat have involved conflict produced by giving it food 
and electric shocks in the same goal box. Specifically, hungry rats are 
first trained to run down an alley to obtain food and are then given 
shocks in the same place. By the principles of classical conditioning, this 
means that the goal becomes simultaneously a positive and negative 
secondary reinforcer, involving conflict. The advantages of this procedure 
are that (1) the separate components of the conflict (hunger-food and 
shock-escape) are relatively easy to control, (2) the behavior of the rat 
in conflict is easily described in physical terms, and (3) this experi- 
mental situation seems to contain many of the features of psychologically 
important conflicts. In particular, human conflict is often of the ap- 
proach-avoidance variety and the resulting behavior is frequently de- 
seribable in terms of nearness to the ambivalent situation or goal. 

In his most recent description of his work on conflict, Miller (1959, 
pp. 201-234) has approached his materials formally, presenting a series 
of postulates and deductions. With minor changes, the postulates are 
reproduced as follows: 
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A. The tendency to approach a goal is stronger the nearer the subject is to it . 
This is an application of the goal -gradient principle described earlier 
(pp. 140). In the description of conflict behavior, it is called an approach 
gradient . 

B. The tendency to avoid a feared object is stronger the nearer the individual 
is to it. This is an extension of the gradient-of-reinforeement idea to avoid- 
ance learning. In this context, the function is called the avoidance gradient. 

C. The strength of avoidance increases more rapidly with nearness to the goal 
than does the strength of approach. In other words, the avoidance gradient 
is steeper than the approach gradient. 

D. The strength of the tendency to approach or avoid varies directly with the 
strength of the drive upon which it is based. In other words, an increase in 
drive raises the gradient throughout its length. 

E. Below the asymptote of learning, increasing the number of reinforced trials 
will increase the strength of the respotise tendency that is reinforced. To this 
it should be added that any of the variables known to influence the strength 
of a learned response should contribute to the strength of the approach or 
avoidance tendency. Such factors as the amount, delay, and scheduling of 
reinforcement have not been thoroughly investigated. What evidence there 
is supports the expectation that these variables will have the same effect in 
the conflict situation ns they do in simple learning (Berkun, 1957), 

P. When two incompatible responses are in conflict, the response corresponding 
to the stronger tendency will occur. This is an application of the assumption 
discussed earlier (pp. 142) in dealing with the results of experiments in- 
volving choices to two paths to a goal. (By permission from Psychology: 

A study of a science, by S. Koch (Editor). Copyright 1959. McGraw-Hill 
Book Company, Inc.) 

Figure 14.2 presents the first three of these postulates in graphic form. 
The baseline in this representation is distance from the ambivalent goal 
situation. The vertical axis is strength of the tendency to approach or 
avoid. The functions themselves are the approach and avoidance gradi- 
ents with the avoidance gradient represented as steeper, as the postulates 
require. The linear form of the functions is purely for the sake of simpli- 
city of representation. Goal-gradient data suggest that the functions are 
actually curvilinear; but the deductions will be qualitatively the same 
from any two gradients which are monotonicnlly decreasing (Miller, 
1959, p. 20G). 

It will bo noted, in Figure 24.2, that the approach and avoidance 
gradients cross. Most of Miller’s deductions involve this set of conditions. 
There are, of course, situations in which the two tendencies are both 
present; hut one is stronger than the other at all distances from the goal. 

In the classical Columhia-obstruclion-box studies, for example, rats ran 



454 


CONDITIONING AND LEARNING 



10 Feared Near Far 


goal DISTANCE 

FIGURE 14.2. Graphical analysis of approach-avoidance conflict. Where the 
approach gradient is stronger, the organism advances toward the goal; where 
the avoidance gradient is stronger it retreats. (This and Figures 14.3 and 
14.4 adapted from Neal E. Miller, “Experimental studies of conflict” in 
Personality and behavior disorders , edited by J. McV. Hunt. Copyright 1944, 
The Ronald Press Company.) 


across an electrified grid to get to food or some other goal object (War- 
den, 1931). This situation contains all the elements of that involved in 
Figure 14.2 except the crossing of gradients. Whether it is to be called 
conflict or something else is entirely a matter of preference. 

Figures 14.3 and 14.4 show the effects which the level of drive and 
the number of reinforcements are assumed to have upon the gradients. 
Figure 14.3 shows the hypothesized effect of two levels of hunger with 
the strength of the avoidance tendency held constant. Figure 14 4 shows 
the effect of varying the strength of the avoidance tendency, with the 
strength of the approach tendency kept constant. These two figures to- 
gether with Figure 14.2, lead to the following five deductions. As with 
the postulates these are taken from Miller (1959, p. 207-208) with only 
minor changes: 


1- The subject should approach part icay to the goal and then stop. Stopping 
should occur at the point where the gradients cross. Nearer the goal, the avoidance 
tendency is stronger and avoidance should occur. Further away, the approach 
tendency is stronger and approach should occur. 


2. Increasing the strength of hunger should 
nearer to the goal. (Figure 14.3) 


cause the subjects to approach 
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FIGURE 14.3, Graphical analysis of approach-avoidance conflict: the effect of 
raising the approach gradient. When the approach tendency increases, the 
point of intersection moves closer to the goal and the animal will approach 
nearer to the ambivalent object. (Adapted from N. E. Miller, 1944). 



FIGURE 14.1. Graphical analysis of approach-avoidance conflict: the effect of 
lowering ihe avoidance gradient. (Adapted from N. E. Miller, 19U). 
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goal DISTANCE 


FIGURE 14.2. Graphical analysis of approach-avoidance conflict. Where the 
approach gradient is stronger, the organism advances toward the goal; where 
the avoidance gradient is stronger it retreats. (This and Figures 14.3 and 
14.4 adapted from Neal E. Miller, “Experimental studies of conflict” in 
Personality and behavior disorders, edited by J. McY. Hunt. Copyright 1944, 
The Ronald Press Company.) 

across an electrified grid to get to food or some other goal object (War- 
den, 1931). This situation contains all the elements of that involved in 
Figure 14.2 except the crossing of gradients. Whether it is to be called 
conflict or something else is entirely a matter of preference. 

Figures 14.3 and 14.4 show the effects which the level of drive and 
the number of reinforcements are assumed to have upon the gradients. 
Figure 14.3 shows the hypothesized effect of two levels of hunger with 
the strength of the avoidance tendency held constant. Figure 14.4 shows 
the effect of varying the strength of the avoidance tendency, with the 
strength of the approach tendency kept constant. These two figures, to- 
gether with Figure 14.2, lead to the following five deductions. As with 
the postulates these are taken from Miller (1959, p. 207-208) with only 
minor changes: 

1. The subject should approach part way to the goal and then stop. Stopping 
should occur at the point where the gradients cross. Nearer the goal, the avoidance 
tendency is stronger and avoidance should occur. Further away, the approach 
tendency is stronger and approach should occur. 

2. Increasing the strength of hunger should cause the subjects to approach 
nearer to the goal. (Figure 14.3) 
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FIGURE 14.3. Graphical analysis of approach-avoidance conflict: the effect of 
raising the approach gradient. When tire approach tendency increases, the 
point of intersection moves closer to the goal and the animal will approach 
nearer to the ambivalent object. (Adapted from N. E. Miller, 1944). 



FIGURE 14. ». Graphical analysii of approach** ofJance conflict? the effect of 
lowering the avoidance gradient. (Adapted from ft. F. Miller, J9M). 
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3. Increasing the number of training trials reinforced with food ( beloiv the 
asymptote) should cause, the subject to approach nearer to the goal. This is be- 
cause increasing numbers of training trials increase the height of the approach 
gradient, with essentially the same effect as increasing hunger. 

4. Increasing the strength of fear by xising a more intense shock should cause 
the subjects to remain farther axvay from the goal. This is because the more in- 
tense shock increases the avoidance gradient. See Figure 14.4. 

5. Increasing the strength of fear by increasing the number of reinforced 
avoidance trials ( below the asymptote ) should cause the subjects to remain 
further from the goal. The reasoning is the same as in connection with the third 
deduction. 

Tests of this theory of conflict have been of two general sorts: (1) 
Those concerned with the validity of the postulates. These have dealt 
with the approach and avoidance gradients separately. (2) Those de- 
signed as tests of the deductions. These have investigated the i nfl uence of 
the two gradients working in combination. The major study in the first 
category is that of J. S. Brown (1948). He trained some rats to approach 
a goal box to get food and trained others to leave it to escape from shock. 
Then, to demonstrate the approach and avoidance tendencies, he meas- 
ured the strength with which the animals would pull against a harness 
either in the direction of the goal or away from it, depending upon 
whether they had been trained to approach or to avoid. The results of 
this study demonstrated the following points: (1) Approach and avoid- 
ance gradients exist as shown by the fact that strength of pull varied 
appropriately with training condition and distance from the goal. (2) 
The avoidance gradient is steeper than the approach gradient. (3) The 
height of the gradient varies with drive, whether hunger or shock- 
avoidance. This study verifies postulates A through D above. Evidence 
supporting the others has come from studies involving actual conflicts, 
rather than just one or the other of the two separate tendencies. 

In an early study of this second sort, Miller, Brown, and Lewis (cited 
in Miller, 1959, p. 212), studied the effects of degree of hunger and 
shock intensity, using nearness of approach to the goal as a measure. 
They found that (1) the animals would go part of the way to the goal 
and stop, and (2) the animals would approach nearer if they were run 
either with strong hunger or weak shock. Subsequently Kaufman and 
Miller (1949) showed that there is a positive relationship between num- 
ber of positively reinforced training trials and the number of animals 
which ran all the way to the goal. Conflict was established by giving 
1, 3, 9, 27, or 81 runway trials reinforced with food and then 3 shock 
trials with a stronger shock on each succeeding trial. Figure 14.5 presents 
a graph showing the number of animals in each group reaching the goal 
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FIGURE 14.5. Number of animals reaching a /eared goal as a joint /unction of 
the number of positively reinforced trials and the number of shock trials 
(Kaufman and N. E. Miller, 1949). 

after 1, 2, and 3 shocks. With increasing numbers of reinforcements it is 
clear that more animals reached the goal following any number of shocks. 
The apparent relationship between number of shocks and number of 
animals reaching the goal depends upon number of shocks and shock level 
jointly, since the intensity of the shock was increased systematically on 
successive trials. 


Displacement and Substitute Behavior 
One of Freud's great insights into human behavior was his recognition 
that much of it is of a substitutive nature. Since society so often prohibits 
direct expression of the urges of the id, these impulses seek indirect out- 
lets which range from such simple expressions as day-dreams and slips 
of the tongue to the most highly constructive and artistic creations. The 
energies of the id arc said to be displaced onto these activities. In terms 
of lenrning theory, displacement can be understood as another conse- 
quence of approach-avoidance conflict. The chief new idea required by 
such an interpretation is that the gradients in question are formed on a 
dimension of stimulus similarity rather than spatial distance. This con- 
ception makes the approach and avoidance gradients special cases of the 
genernlirntion gradient. Both Spence (1917) and Miller (1959) have 
made a case for this interpretation even for spatial gradients. 

To illustrate the application of conflict theory to displacement phe- 
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nomena, consider tlie consequences of punishment as it is involved in 
child-rearing practices. A child, let us say, is punished physically for 
some misdeed. Since punishment serves to frustrate some line of moti- 
vated activity there follows a strong, probably learned, tendency to 
aggression directed at the parent who does the punishing (Dollard, Doob, 
Miller, Mowrer, and Sears, 1939). The aggression, however, does not 
materialize, because the punishment has also led to the conditioning of 
a fear of the parent. But it may appear, seemingly with little or no 
provocation, in some other situation typically against a sibling to whom 
the aggression generalizes. The concept of generalization is basic to the 
S-R interpretation of displacement. 

Miller and Kraeling (1952) have performed an experiment designed 
to test this interpretation of displacement, using the straight-alley con- 
flict situation previously described. In this experiment, rats were first 
trained to approach and obtain food in an alley of a distinctive color 
and width. After this habit had been thoroughly learned, the animals 
received increasingly strong shocks at the moment of touching the food 
until they refused to take it. At this point it was assumed that a conflict 
had been established. In order to study the phenomenon of displacement, 
different groups of rats were tested either in the original alley or in one 
of two others which were increasingly different from the original alley. 
In this test only 23 per cent of the animals tested in the original alley 
went all the way to the goal box. Instead they showed the typical behavior 
of the rat in conflict in this situation. They went only part of the way to 
the goal and stopped. Thirty-seven per cent of the animals tested in the 
alley of intermediate similarity to the training alley reached the goal, 
whereas 70 per cent of those tested in a very different alley responded 
in this way. In short, as the test situation became more and more differ- 
ent a greater and greater proportion of the subjects performed the pun- 
ished response. 


Evaluation of S-R Conflict Theory 

Prom the foregoing discussion, it is clear that the main postulates in 
the stimulus-response theory of conflict have been verified, as have some 
of the more straightforward deductions. The theory has also been ex- 
tended with some success to situations more nearly resembling real life 
(Miller, 1948b; Murray, 1954; Murray and Berkun, 1955; Whiting and 
Child, 1953). On the other hand, S-R conflict theory as presented so far 
has certain shortcomings which presumably will have to be remedied 
before a completely adequate translation can be made. As is true of 
other extensions of S-R theory, concepts designed to handle symbolic 
behavior have been introduced relatively recently. Whether these at- 
tempts will be successful remains to be seen. Such developments seem 
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particularly important in connection with the topic of human conflict 
because the competing response tendencies are usually internalized 
(intrapsychic) and involve a large element of symbolic representation. 
Presumably because of the central importance of this problem it has 
been one of the first to which proponents of this kind of conflict theory 
have addressed themselves (Bollard and Miller, 1950). 

Other problems involve the details of the theory. The following 6et of 
examples provides a sample: (1) One of the basic assumptions in con- 
flict theory is that the avoidance gradient is steeper than the approach 
gradient. Why should this be? One possibility (Brown, 1948 ; Miller and 
Murray, 1952) is that the internal stimuli arising from hunger make the 
stimulus situation more nearly the same at various distances from the 
goal than it is in the case of fear. Although this argument seems to have 
some merit for the hunger-fear conflict used in experimental studies with 
lower animals, it seems less plausible when extended to human behavior. 
(2) Conflict seems to have a particularly disruptive effect upon behavior. 
Why is this? Specifically it has been demonstrated (Masserman, 1943) 
that a fear-conditioning situation alone produces avoidance, but does not 
produce the symptoms of experimental neurosis. Apparently the conflict 
between fear and hunger is critical. It may be that the effect of conflict 
comes about through the addition of drives, but this seems unlikely. It is 
more probable that conflict itself is a negative drive and that learning 
to avoid conflict occurs. Little, if any, research has been done on this 
problem. Guthrie’s (1938) interpretation suggests that conflict may 
have its effect by keeping the individual in a constant state of heightened 
tension resulting in physical exhaustion. (3) Conflict, so far, lias been 
described grossly in terms of competin' r reaction tendencies, a descrip- 
tion which eventually will have to be refined (Ilaner and P. A, Brown, 
1955). The concept, reaction tendency, is a performance concept} and we 
have seen that performance is jointly determined by habit and (mainly) 
motivational mechanisms. From this, it follows that conflicts between 
equally strong reaction tendencies may come about in a variety of wars. 
They may represent conflicts of motives, as the introductory text books 
usually imply; they may derive from strong competing habits; or they 
may occur in situations where one tendency involves a weak habit oper- 
nting under intense drive, and the other involves a strong habit and 
weak drive. No doubt the influences of these different forms of conflict 
upon behavior will differ. The implications for eliminating the conflict 
differ even more, since the procedures required for reducing motives and 
extinguishing habits arc essentially opposites of each other. For « hunger- 
motivated response, one feeds the animal to reduce the drive and fails 
to feed it to destroy the habit (Kimble, 3956). It seems very likely 
that the concept of conflict might profit from being thought of in quanti- 
tative terms. Employing Miller’s graphical analysis of conflict, it is 
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possible to construct a great variety of approach and avoidant gradients, 
all of which lead to the prediction that the rat will stop (say) halfway 
to the goal. This can be accomplished by raising and lowering the heights 
of the avoidance gradients together, so that the point of intersection does 
not change. It seems quite likely that the conflict produced by two strong 
tendencies might have very different behavioral consequences from one 
produced by two weak ones. 

THE ORIGIN OF MALADJUSTMENT 

At one level or another, all theorists regard behavior as adjustive in the 
sense of satisfying needs and resolving conflicts. Most human behavior is 
quite well suited to the attainment of these objectives. But there are some 
people whose behavior is much less than adequate. There are also cases 
in which the behavior of the well-adjusted person is maladaptive. This 
section deals with the attempts which have been made to understand 
such behavior in terms of learning theory. 

Symbols, Language, and Cue-Producing Responses 

The main problem which the learning theorist faces in attempting to 
understand complex human behavior is that such activity is mediated. 
That is, between the occurrence of a stimulus and the response to it, 
there intervenes a series of internal events which are subtle and difficult 
to investigate. The individual “plans his line of action,” “thinks about 
what to say next,” “decides whether or not to respond,” “wonders 
whether he is on the right track,” “feels guilty about wanting to do the 
wrong thing ’ ’ — all covertly, with little or no visible sign that all of this 
is going on within him. Much, but not all, of this intervening behavior is 
verbal. 

Because of the intimate connection between verbal behavior and ad- 
justment, it will be necessary, before proceeding further, to present a 
brief outline of a psychology of language which employs the concepts of 
conditioning. It seems useful to begin by noting that the distinction be- 
tween classical and instrumental conditioning applies to verbal behavior. 
Language, on the one hand, is a complex set of responses used by the 
individual to provide cues (communicate) to others. On the other hand, 
it is also a set of stimuli to which one has learned to respond. With re- 
spect to the former (response) function, language seems to develop in 
the same way as other instrumental acts (Skinner, 1957). Certain com- 
binations of words and intonations of voice are strengthened through 
reward and are gradually made to occur in appropriate situations by 
the processes of discrimination learning. 

In the other (stimulus) function of language, the principles of classical 
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and instrumental conditioning both seem to apply. Words supply cues 
to action and, in this case, serve as discriminative stimuli in a practical 
application of the procedures of instrumental learning. W T hen we are 
told to “hurry up,” we probably do so because in the past we have been 
rewarded for such behavior and punished for failing to obey. But words 
also have signal and symbol values which seem to result from their 
having occupied the position of conditioned stimuli in practical analogues 
to a classical conditioning experiment of the higher order sort. As a 
result, words acquire the same cue and secondary reinforcing functions 
as the stimuli in any classical conditioning study. Pavlov (1927) reported 
that one of his dogs, conditioned to salivate to a light, licked the light as 
if it were food. Zener (1937), in contrast, pointed out that such be- 
havior is not at all common and that more often animals respond to the 
CS with a response in preparation for the occurrence of the UCS. Today 
it seems quite likely that Pavlov's observation and Zener's were both 
correct. The behavior described by Pavlov appears to be an unusually 
direct reflection of the incentive value of a CS. The more typical be- 
havior described by Zener reflects the cue function of a CS. Words, be- 
cause of a history of association with significant events, acquire the 
properties of incentives and secondary reinforeers, typified by the terms, 
“good,” “bad”, and their synonyms. They also acquire signal proper- 
ties. When the dentist says, “This is going to hurt a little,” we respond 
by habitual actions in preparation for pain. 

Thinking Repression and the Unconscious 

Just as language can supply cues to guide the behavior of other people, 
it is also a means by which one can direct his own responses, for example, 
when one verbalizes the steps in a problem. Similar behavior occurs in 
lower animals, overtly, in the form of vicarious trial ami error. In human 
behavior it, more often, occurs covertly as wlint wc call thought. The 
responses involved in thinking may be of any sort, but they are pre- 
dominantly verbal. Among the important functions of thought is that 
of supplying cues to subsequent behavior. 

In childhood, thinking often occurs aloud and tlte relationship between 
it and language is quite clear. Confronted with n problem, the three- 
year-old may verbalize his solution (“Maybe I could stand on n chair 
to get it.”). Ape ami practice, however, lead to two significant changes: 

(1) The ovet verbalization disappears, although the responses may 
continue to oveiir subvocally. This is probably partly because the child 
learns that certain plans, such ns competitive ones, and certain feelings, 
Mich as those of guilt or smugness, are better left unsaid (Guthrie, 193*) 

(2) As with other sequences of behavior, the verbal chain tends to lx* 
shortened and compressed. Thought is as rapid as it sometimes is because 
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of this feature. The complex sequence of verbal items is replaced by 
some fragment. The advantages of this lie in a saving of time and energy . 

The fact that thoughts are capable of providing cues means that, 
theoretically, they can provide the same conditions for the control of 
behavior as environmental events do. At a street intersection the thought, 
“I must either go straight ahead or turn left, ” has exactly the same effect 
as does a sign which reads “no right turn.” From the same general fact 
it follows that thoughts can be punishing or rewarding if they have been 
associated with rewards or punishments in the past. The most important 
application of this idea is to the Freudian concept of repression, which 
refers to the inability to remember some traumatic event. 

Personality maladjustment always contains an element of conflict 
which is unverb alizable or unconscious. In part, this is because the con- 
flicts often have occurred before the age of talking, making clear repre- 
sentation in thought impossible (see below). But it is also partly a 
matter of repression. A person in a highly dangerous or guilt-provoking 
situation responds to it in many ways, with the skeletal musculature, 
autonomically, and verbally. These responses produce stimuli which then 
may become conditioned to the anxiety evoked in the situation. On later 
occasions talking about the frightening experience, or even thinking 
about it, leads to the reproduction of these stimuli and therefore to 
anxiety. The only response which prevents the uncomfortable arousal of 
anxiety is to cease thinking about the unpleasant occasion. If one does 
discontinue such thoughts, reinforcement in the form of anxiety reduc- 
tion follows and strengthens the tendency not to think of the topic. With 
time, the tendency to repression spreads until all of the stimuli which 
might serve as a reminder only evoke avoidance of the unpleasant 
thought. No question, hint, or suggestion will stimulate recall. What is 
lost in repression is the ability to produce any of the responses which 
provide the stimuli conditioned to intense anxiety. 


The Development of Neurosis 

The materials presented up to this point contain the main principles 
necessary to describe neurotic maladjustment in terms of learning theory. 
Such disorders have their origin in conflict which remains unresolved, 
producing great anxiety. Since the elements of the conflict are repressed 
and unconscious, the neurotic person cannot form a reasonable plan of 
action to cope with his discomfort. Conflicts of this sort occur, of course, 
in normally adjusted individuals, too. The difference between normal 
and neurotic conflict appears to be partly the result of the stimuli in 
control of action. In neurotic conflict, these stimuli seem to be produced 
by aspects of the person’s own behavior. To illustrate the importance of 
this point, consider sexual tendencies, which are so often involved in 
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neurotic behavior. The monograph of Ford and Beach (1952) suggests 
that, in normal male sexual behavior, the most important factor is the 
availability of an attractive female. That is, the strength of instigation 
to sexual behavior depends upon the presence of a sexual goal object. In 
neurotic behavior, however, there is often a preoccupation with sex which 
indicates a control of sexual tendencies by other factors, probably in- 
ternal and responses produced. This means that the neurotic person can 
carry his conflict around with him in a way which the normal individual 
does not. It is possible to conceive of two individuals with equally strong 
urges in the direction of socially prohibited sexuality and with equally 
strong guilt reactions providing restraint. For one of these individuals, 
the urge arises only with opportunity because it is controlled by external 
stimuli; for the other, however, it is constantly present because it is 
called forth by proprioceptive stimulation. For the first of these in- 
dividuals, anxiety arises only in a few situations; for the other, it is a 
continual problem. 

If almost any person is put into a highly conflictful situation and 
forced to remain there, the equivalent of neurotic maladjustment may 
occur under conditions where the conflicting action tendencies are under 
external stimulus control. War neuroses develop under circumstances 
where there is intense conflict between fear and the demands of battle 
(drinker and Spiegel, 1945). Normally a person avoids such situations 
and leaves them if he can. The inescapability of the battle situation is one 
of the conditions leading to neurosis. The conflicts of other neuroses arc 
also inescapable, but for a different reason, because the elements of con- 
flict are in the individual’s behavior. 

Another important difference between neurotic and other conflict is 
that the former is usually unconscious, at least in part. This comes about 
for three reasons; (1) Anxiety reactions are acquired by a process of 
classical, rather than instrumental, conditioning. This means that the 
response is elicited rather than emitted and, being out of voluntary con- 
trol, Jess likely to contain an dement of verbalization in the first place. 

(2) Anxiety-producing responses, including memories and thoughts, 
tend to be repressed for the reasons developed earlier. (3) The traumatic 
situations responsible for neurotic conflict are often unverbnlizablc, re- 
pressed events in the distant past. Among the earliest adaptations re- 
quired of the child are those connected with feeding himself, keeping 
clean, and avoiding expressions of aggression and sexuality. It is neces- 
sary to eat at certain times, in certain places, and in certain way* called 
"manners.” One must learn to bring the sphincters controlling elimina- 
tion under voluntary control, a rather considerable physical accomplish- 
ment. Aggression must he inhibited or modulated. And sexual impulses 
must be curbed. These are hard lessons for the child, learned under 
punishment or threat of punishment and resulting in a groat deal of 



Although the importance of presumably innate tempermental factors 
should not be underemphasized, it seems quite likely that situational 
conditions and the history of the individual are more important in 
determining the form that symptoms take. Conflict, it will be recalled, 
is a condition in which goal-directed activity is frustrated. When frus- 
tration occurs, the frustrated response must be abandoned and another 
substituted for it. Sometimes the new response will be an earlier response 
employed in the same or in a similar situation. For this reason responses 
to frustration, and therefore neurotic symptoms, are often regressive. 
The process of growing up, as we have seen, involves a series of conflicts 
between the urges of the child and the requirements of society. At each 
stage in development, more mature adjustment consists of replacing an 
older habit with a newer and better one. Acquiring the new habit, how- 
ever, does not mean that the old one is completely obliterated. It still 
exists as an alternative to the new behavior and may appear in times of 
stress. In Freudian terms, the individual remains fixated at a certain 
level of development. In the terms of learning theory, the old habit 
remains as a part of the habit-family hierarchy. Numerous studies (Ham- 
ilton and Kreehevsky, 1933 ; Sanders, 1937 ; Martin, 1940; Mowrer, 1940) 
have demonstrated regression to former habits in lower animals. Martin, 
for example, trained rats to make a choice in a T-maze and gave three 
different groups three different numbers of trials on this habit. Then he 
reversed the problem and required the rats to learn to take the opposite 
turn. In the course of learning this second habit, a shock was adminis- 
tered at the choice point. As a result, many of the animals showed a 
tendency to regress to the first habit. The proportion regressing varied 
directly with the amount of training on the first response. This is exactly 
as Freud had said *. The stronger the fixation in the path of development, 
the more easily the function yields before external obstacles by regressing 
onto those fixations. 

The behavior of the human organism under stress differs from this in 
being much more complex, because of the wide array of habits at his 
disposal. Early in the development of neurosis, there is often a profusion 
of symptoms which appear one after another, only to be abandoned. In 
time, the symptom picture becomes stable and most of the symptoms dis- 
appear permanently, leaving behind only a few. The reason for this 
apparently is that the symptoms remaining, however inadequate, are 
those which are most consistently reinforced. The reinforcement is always 
a matter of lowered anxiety, at least in part. Eut sometimes secondary 
gains are a contributing factor, as in the case of the neurotic veteran 
whose pension is dependent upon the continuance of his symptoms 
(Dollard and Miller, 1950, p. 196). Once the symptom has been estab- 
lished, it tends to spread by what appears to be a complex application of 


which a compulsive fear of the number 13 gradually spread to the thi: 
teenth tread of a stair, the thirteenth day of the month, and salutatior 
containing thirteen letters. 


PSYCHOTHERAPY 

Early applications of conditioning principles to psychiatric problem 
employed direct applications of conditioning technique to diagnosis am 
treatment. The detection of sensory function in cases of hysterical anes 
thesia by such methods has been carried out successfully for a long time 
Bekhterev early reported his success with hysterical deafness (1912), ant 
his procedure "was confirmed later by Myasishchev (1929). R. R. Sear 
and Cohen (1933) studied a case of hysterical anesthesia, analgesia, ant 
astereognosis by conditioning methods. They found that when conditionec 
responses were established to stimulation of supposedly anesthetic areas 
the anesthesia disappeared. Cohen, Hilgard, Wendt (1933) were able tc 
demonstrate visual sensitivity in a patient with hysterical blindness, bul 
the symptoms did not disappear as a result of conditioning. 

In addition to being used in diagnosis, conditioning was occasionallj 
used in therapy. The following case illustrates the kind of clinical appli- 
cation which is feasible. The patient, an unmarried school-teacher of 32, 
had been in a motor car accident six years before coming for treatment, 
During these six years her left arm had been entirely useless to her, 
being totally paralyzed and anesthetic. Neurological examination showed 
the anesthesia to be of the glove type, with a line of demarcation at the 
shoulder. The symptoms were thus shown to be functional. The arm and 
band showed considerable atrophy from disuse. Attempts by a psychia- 
trist to give the patient insight into the functional nature of the ab- 
normality met with great resistance, although the fact that the patient 
had returned to the hospital voluntarily in the hope of a cure was con- 
sidered a hopeful sign. The psychiatrist arranged to have the patient 
treated by conditioning methods in the psychological laboratory which 
adjoined the psychiatric unit. 

A finger-withdrawal experiment was arranged. Two electrodes were 
used, one for each hand. The first series of experiments consisted in 
presenting a shock to the anesthetic hand as the conditioned stimulus and 
a shock to the normal hand as the unconditioned stimulus. This was 
designed to give evidence of sensitivity in the anesthetic hand, since a 
shock to it served as the signal for withdrawal of the normal hand. Al- 
though little conditioning occurred, the desired effect was produced and 
sensitivity gradually returned in the insensitive hand and arm. Experi- 
ments were repeated daily, and the improvement was gradual. After 
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recovery of sensitivity, the conditioning procedure was reversed for pur- 
poses of developing voluntary control. The normal hand was given a 
light shock which served as the conditioned stimulus. For the uncondi- 
tioned stimulus, the paralyzed hand was given a more severe shock, to 
which it was now fully sensitive. Presently movement began to occur in 
the paralyzed hand at the signal given to the normal hand. This was 
the beg innin g of control, and voluntary movement was gradually re- 
stored. At this stage, the patient was given physiotherapy to strengthen 
the muscles which had been so long unused. The symptoms had not re- 
turned two years later, nor were any additional symptoms reported. 

This case illustrates the feasibility of conditioning procedures in the 
treatment of psyehoneurotie symptoms. The experimental situation was 
one that permitted the patient to eliminate her symptoms without loss of 
face. She believed the machine to be making the cure. Although the condi- 
tioning procedure was the effective means for bringing the symptoms to 
an end, other dynamic factors in the situation were probably of great 
importance. In most cases, it is necessary to treat basic mechanisms, as 
well as symptoms, in order to effect a permanent cure. 


Obstacles to Psychotherapy 

From the point of view of the learning theorist, maladjustment con- 
sists of a set of inadequate habits designed to reduce the anxiety associ- 
ated with conflict. From the same point of view, therapy is a matter of 
finding other ways of reducing anxiety. As we shall see, there are several 
alternative possibilities. Before considering these, however, we may note 
that the translation into the concepts of learning contributes to an under- 
standing of the great difficulties often associated with the achievement of 
a psychiatric cure. 

We have previously seen that learned behavior does not spontaneously 
disappear with rest. Extinction occurs only when there is some active 
interference with behavior. In the case of the responses involved in per- 
sonality disorder, the difficulty of extinction is increased by the fact that 
the behavior to be eliminated is based upon fear. In at least some quarters, 
there is a disposition to question whether fear can ever be eliminated, and 
it is certain that fear can be very resistant to extinction. This means (1) 
that therapeutic tactics must, in the main, be directed at the response 
rather than the motive behind it, (2) that the original motive responsible 
for maladjustment remains and often creates new symptoms as the orig- 
inal ones are destroyed, and (3) that society’s main disciplinary agent, 
punishment, will probably be of little value in treating personality diffi- 
culties. Since they are based on fear in the first place, punishment,* which 
also elicits fear, only provides the occasion for the punished response. The 



APPLICATIONS TO PERSONALITY 


469 


experimental evidence supports this line of reasoning. When punishment 
is accompanied by a clearcut alternative response leading to the original 
goal, it sometimes hastens the abandonment of the punished response 
(Whiting and Mowrer, 1943), In cases where no available response leads 
to the original goal (Estes, 1944) punishment apparently does not ex- 
pedite extinction. And, avoidance responses based upon shock (Gwinn, 
1955) become more persistent when animals are shocked for performing 
them. This suggests that the therapist must either (a) resort to other 
motive-reinforcement combinations, or (h) attempt to reduce anxiety, 
realizing that in some cases this may turn out to be impossible. 

Another set of conditions standing in the way of successful psycho- 
therapy involves the cues controlling maladaptive behavior. For one thing 
these cues are very pervasive. Through a process of stimulus generaliza- 
tion, one’s behavior tends to extend itself to all areas of life. This is what 
makes the individual’s responses predictable and dependable. When the 
aim is to alter behavior, however, this fact also makes it very difficult to 
eliminate the responses in question. As Guthrie (1935) has stressed, it is 
necessary to disconnect an act from all of the stimuli which control it 
before it can be regarded as completely extinguished. The therapist has 
one ally in the accomplishment of this formidable objective, namely, the 
generalization of extinction. Responses eliminated in one situation should 
show a diminished tendency to appear in others. It will be recalled, how- 
ever, that the generalization of extinction can be counted on only for some 
weakening of the response, not for its complete obliteration. Moreover, if 
it is correct that the generalization gradient for extinction is narrower 
than that for excitation, this limits the usefulness of the mechanism. 

Still considering stimulus variables related to the accomplishment of 
psychotherapy, it is important to recognize that these are internal, subtle, 
minimal cues which are difficult for the patient to recognize. Accordingly, 
one of the main goals of psychotherapy is to get the patient to recognize 
these cues. This requires that he be forced to recall all-but-forgotten past 
experiences and to label the dimly recognized impulses which supply the 
eues to the undesirable behavior. Psychotherapy must make these cues 
more distinct and conspicuous in order to facilitate the conditioning of 
new and better responses to them. The task of making these cues available 
to the patient is impeded by the process of repression. Recalling a trau- 
matic experience or recognizing a tabooed impulse calls up eues associated 
with powerful anxiety reactions. Thus, as the therapist gets close to 
significant problem areas, anxiety often becomes acute and much evasive 
behavior ensues. The patient forgets appointments, decides that his case 
is hopeless, or finds that he has to visit a relative in a distant city. In each 
case the behavior is calculated to escape from the therapeutic situation 
where these anxiety-evoking stimuli are apt to appear. 
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Therapeutic Possibilities 

Looked at as a problem in learning, psychotherapy becomes a matter of 
equipping the patient with responses to relieve the misery he experiences 
as a result of conflict and anxiety. Even considered abstractly, it is clear 
that the problem is one of enormous complexity. In such terms there are 
four possible courses of action to be considered. The possibilities emerge 
from the fact that neurotic conflict is of the approach-avoidance variety, 
and the approach and avoidance components each comprise a habit and a 
drive factor. Thus the four conceivable lines of therapeutic action are the 
following : 

1. Believe the intensity of the drive toward the positive goal. Masser- 
man (1943) makes the point that the cats in his studies of experimental 
neurosis showed little or none of their neurotic behavior if they were 
satiated when placed in the apparatus. Speaking in terms of conflict this 
means that the approach gradient was lowered to the point where no con- 
flict existed. To the extent that this is possible in cases of human conflict, 
it represents a constructive point of departure. "Where conflict involves 
unrealistic aspirations, say, to enter a scientific or other exacting career, 
it may be sound therapeutic tactics to try to get the person to abandon his 
unrealistic goals. In eases of sexual conflict, on the other hand, it seems 
unlikely that reducing the level of sexual drive would be practical, be- 
cause of the anxiety which would be associated with almost all available 
outlets. 

2. Alter the habits mediating attempts to attain the positive goal. The 
behavior of the neurotic is ill-considered and impulsive. It often lacks the 
elements of hesitation and restraint which are characteristic of normal 
adjustive behavior (Guthrie. 1938). Or it sometimes is over-cautious. In 
either case, one aspect of therapy obviously must be to provide the patient 
with better means of dealing with his desires. 

3. Reduce the intensity of conflict by reducing anxiety. Implicitly or 
explicitly, traditional psychotherapy seems to have concentrated on this 
possibility. The patient is urged to recall and relive traumatic experi- 
ences. In this way anxiety is elicited and subjected to extinction. As the 
therapist well knows, one implication of this procedure is that painful 
anxiety must occur during therapy if a cure is to occur. This fact is 
responsible for the main practical problems which arise in psychotherapy. 
As we have seen, the patient goes to great lengths to prevent having such 
anxiety aroused in him. The learning theorist might also raise a question 
about this procedure on other grounds. If it is correct that very traumatic 
anxiety is inextinguishable, it would be impossible to eliminate certain 
conflicts by reducing the avoidance gradient. 

4. Alter the habits mediated by anxiety. One of the main points here 
is that the complete elimination of anxiety, even if it were possible, would 
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be undesirable. Anxiety provides the individual with a set of moderating 
influences which control normal behavior, and it would be a mistake to 
rob the patient of these controls. What he needs instead is some more 
constructive means of handling his anxiety. As with the habits mediated 
by the patient's positive motives, it is important to give him a method for 
dealing with his fears. 

As a final point in connection with these brief comments on therapeutic 
possibilities, it should be mentioned that they are merely theoretical 
alternatives. Whether they are all equally useful, either in individual 
cases or as a general rule, is something which only more detailed analysis 
of the psychotherapeutic process can determine. It seems quite clear that 
the practicing therapist uses all of them, employing at every point the 
technique which his evaluation of the situation suggests. 


Drive, Cue, Response, Reward 

The most complete treatment of psychotherapy in learning terms is 
that of Dollard and Miller (1950) who deal with the problem along the 
same lines as they used in their analyses of social behavior (Miller and 
Dollard, 1941). The essentials of this analysis may be presented quickly: 
For learning to occur, a response, stemming from an important drive , 
must take place in the presence of a relevant stimulus or cue; reward, in 
the form of drive-stimulus reduction, must follow immediately. The para- 
digm is that of instrumental reward learning, and it is assumed that 
psychotherapy can be conceived in these terms. 

1. Drive. The motives for therapy are the same as those which underly 
maladjustment, the powerful drives comprising the conflict and, in addi- 
tion, an element of misery coming from the conflict itself. Since these mo- 
tives are the same as those which originally produced the symptoms, a 
great deal depends upon how effective the symptoms have been. If they 
have proven quite successful, the patient ’a drive level may have been re- 
duced to a point where there is little motivational basis for a cure. 

2. Reward. Reinforcement for the new response also derives at least in 
part from the same mechanism as that which led to the development of 
the neurosis. For therapy to be successful it is necessary for the new 
responses to be more satisfying than the symptoms. Otherwise the laws 
of reinforcement indicate that the sjunptoms will persist. In this con- 
nection, considerations extraneous to maladjustment are often important. 
A person from the upper social and economic levels, one who is natively 
intelligent and is physically attractive, usually obtains greater benefit 
from improved psychological adjustment than one who does not possess 
these advantages. Conversely, patients who derive secondary gains (such 
as pensions) from their symptoms have less to gain from successful treat- 
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ment. For these reasons, psychotherapists use such criteria in deciding 

■whether or not to accept a given patient. 

3. Response. In the final analysis, the patient must find the new and 
more adjustive behavior within himself. Some forms of therapy, nsnally 
called non-directive (Rogers, 1942), stress this fact particularly. In other 
forms of therapy, the therapist takes a more direct hand in suggesting 
lines of action. He may encourage the patient to east around for his own 
solution to problems ; but, at the same time, he makes suggestions and calls 
attention to the examples of what others have done. He tries to get the 
patient to test his new and tentative adjustments, first in safe situations 
such as his office and then in nondangerous outside situations. After at- 
tempting to build confidence and to reinforce appropriate behavior in 
these situations, the therapist relies on generalization to carry it to still 
wider application. The method is not unlike the method of successive 
approximations employed in the preliminary training in many animal 
experiments. 

4. Cue. From the point of view of learning theory, the most difficult 
problems of psychotherapy involve the stimulus concept. This is because 
the cues in neurotic conflict are internal, response-produced, and hard to 
isolate. Such problems develop in a number of different ways. First, the 
problem of leading the patient to understand his conflict seems mainly a 
matter of getting him to recognize and to label certain urges and fears 
which hitherto have been unelearly perceived, if they have been perceived 
at all. The patient must bring to the present events which were poorly 
understood and unlabeled when they occurred. Naturally, this is hard to 
do. Once recalled, however, it is usually possible to label these memories 
in ways which make them less threatening. Next the patient, having 
recognized the problem, must be made to distinguish between word and 
deed, to see that having a certain thought is really quite a different thing 
from translating the content of that thought into action. Simple as this 
seems, it should be recognized that thoughts are fractions of acts, and this 
may make the discrimination difficult. 

Finally, the whole success of therapy depends upon considerations in- 
volving the stimuli which control the patient’s behavior. The basic ques- 
tion is this: How can solutions arrived at in mere consultation have any 
effect upon actions outside the therapeutic situation? Obviously there 
must be some way in which therapy provides the patient with a set of 
cues by means of which he transfers his new learning from the therapist’s 
office to real life. What seems to happen is that the patient comes to re- 
spond to threatening situations with mediating responses (initially verbal 
labels) which produce the cues to adaptive behavior. Situations which 
once evoked anxiety and its stimulus consequences now produce labeling 
responses which set off more appropriate action. The result is sometimes a 
stage of too deliberate, unspontaneous, overly intellectualized adjustment 
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in which the patient plans his actions with care, expresses alternatives, 
and verbalizes his emotions. With practice these mediationa] links tend 
to shorten and to disappear. 

Evaluation of the Learning Theorist’s Analysis of Therapy 

It is important, finally, to make this point : Learning theory is not a 
theory of psych otherapy. It is a position which attempts to interpret 
simple learning. The analyses in this chapter have extended certain of the 
theory's experimentally derived concepts in what appear to be suggestive 
directions. Whether such extension is fruitful will depend upon two 
things: (1) Whether there is any sense in which this analysis produces a 
valid description of personality processes. This is a matter which experts 
in the fields of personality and psychopathology will be able to judge. 
And (2) whether the analysis leads to effective ways of dealing with 
personality problems in experimental and practical, psychotherapeutic 
terms. It is still too early to evaluate the interpretation in terms of 
learning theory from this point of view. 

SUMMARY 

One of the basic questions connected with the study of conditioning 
concerns the range of applicability of the principles discovered in these 
simple learning situations. In this chapter we have explored the possi- 
bility of applying these principles to materials in the field of personality. 

Freud’s Contribution 

Somewhat surprisingly there are numerous parallels between the ap- 
proach to the problem of personality taken by the learning theorist and 
that of the psychoanalyst. Among" these parallels are the following: (1) 
Psychoanalytic theory, although not itself a theory of learning, is his- 
torical in orientation. Thus it shares with learning theory a common 
initial commitment. (2) Freudian theory contains, in the concepts of 
primary and secondary process, ideas which seem potentially translatable 
into the basic learning mechanisms of classical and instrumental condi- 
tioning. (3) Psychoanalysis also contains, in figuratively presented form, 
the essentials of such learning phenomena as acquisition, stimulus gener- 
alization, and acquired dmc. 

Experimental Neurosis 

Another contribution to the learning theorist’s interest in personality 
stems from the laboratory investigation of experimental neurosis. It was 
originally discovered in Pavlov's laboratory that certain dogs developed 



474: CONDITIONING AND LEARNING 

strange and seemingly maladaptive behavior if they were forced to make 
too fine a discrimination. Subsequently other procedures for establishing 
experimental neuroses were also discovered. In the terms of Pavlov’s 
theory, any circumstance which evoked unusually strong excitatory or 
inhibitory responses, or led to a clash of excitation and inhibition, could 
produce an experimental neurosis. Later experimentation was successful 
in developing experimental neuroses in animals of other species. The dis- 
orders reported as being characteristic of experimental neurosis have in- 
cluded a variety of motor, autonomic, emotional, and social symptoms. 


Studies of Anxiety 

A condition basic to maladjustment, which was stressed by Freud and 
is also apparent in eases of experimental neurosis, is anxiety. For the 
learning theorist, anxiety is the conditioned form of reactions to painful 
stimuli. As a result of this view, investigations of anxiety in the labora- 
tory have been largely limited to what Freud called objective anxiety, 
that is, anxiety about real, physical situations. The learning theorist 
recognizes, however, that anxiety may be conditioned to drive stimuli or 
to response-produced stimuli. Anxiety conditioned to such stimuli may 
correspond to the psychoanalytic processes of neurotic and moral anxiety. 

One advantage of the view that anxiety is a conditioned pain reaction 
is that it leads to the search, in the life history of the individual, for 
traumatic occasions which may provide the origin of anxiety. This has led 
to the emphasis on certain events in childhood which are often responsible 
for the development of anxiety. These events include some of the same 
episodes stressed by the psychoanalysts in their theory of psychosexual 
development, such as toilet training and the inhibition of infantile 
sexuality and aggression. One of the important points about such fears is 
that they are acquired before the child can talk. This appears to con- 
tribute to the difficulties encountered later on in trying to remember 
traumatic events which led to the development of anxi ety. 

Experimental studies of the development of anxiety in lower animals 
were discussed in the last chapter. In this chapter we have concentrated 
on studies of the functioning of anxiety in human subjects. Much of this 
research has employed the strategy of selecting subjects with very differ- 
ent levels of manifest anxiety and studying the differences in learning 
displayed by such subjects in a variety of situations. The assumption be” 
bind many of these investigations is the familiar one that anxiety is a 
drive and should have the same effect upon behavior as is obtained by 
manipulating other drives. The results of relevant experiments support 
this assumption in a general way ; but, as with other motives, it turns out 
that a er% important consideration involves the responses elicited by 
anxiety. 
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Conflict 

Conflict occurs when incompatible action tendencies are instigated 
simultaneously. Although many different varieties of conflict are de- 
scribed in the literature, the kind which is of greatest interest in a discus- 
sion of personality is approach-avoidance conflict, which is basic to much 
human maladjustment. An approach-avoidance conflict is one in which 
the same goal object elicits tendencies toward approach and avoidance at 
the same time. At the human level such conflicts often center about sex 
and aggression. 

Stimulus-response psychology has made an effort to deal with approach- 
avoidance conflict in terms of a formal theory. In this theory both the 
approach and the avoidance tendencies are conceived as depending upon 
the conventional variables studied in experiments on learning. They are 
also considered to increase in strength with increasing proximity to the 
ambivalent goal object. These assumptions have been demonstrated to be 
capable of handling certain elementary forms of conflict, particularly in 
experiments with lower animals. Application at the level of human be- 
havior has just begun. 


The Importance of Language 

One of the chief difficulties involved in any attempt to deal with human 
activity in terms of the concepts provided by simple conditioning experi- 
ments stems from the fact that so much human behavior involves language 
as a mediating link between stimulus and response. This is not to say that 
such an account is impossible. Applications of S-R theory to verbal be- 
havior stress the fact that words are cue-producing responses. These 
responses acquired from childhood on are, at first, overt. With age, how- 
ever, the responses become covert and occur as thoughts. 

The fact that words (and, therefore, thoughts) are cue-producing re- 
sponses makes it potentially possible to understand such difficult concepts 
as repression and the unconscious in terms of learning theory. Repression, 
for example, becomes a case of avoidance learning in which one acquires 
the habit of avoiding certain thoughts because the stimuli provided by 
these thoughts are the occasion for intense anxiety. The avoidance re- 
sponse of not thinking about the topic is reinforced by anxiety reduction. 


Maladjustment and Psychotherapy 
The materials presented so far contain the main principles necessary 
to describe maladaptive behavior in S-R terms. The important elements 
are conflict, anxiety, and the inability to deal with either of these by 
means of reason (repression). The result in some people is a break from 
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conventional modes of adjustment and the development of neurotic symp- 
tomatology in their place. These symptoms are learned because they pro- 
vide occasional and temporary relief from conflict and anxiety. They are 
maladaptive because, besides being unconventional, the relief they bring 
is neither consistent nor permanent. 

The analysis of maladjustment in terms of conflict, anxiety, and re- 
pression points the direction in which therapy might proceed. Its method 
might be to bring traumatic events to consciousness so that the fear they 
evoke might extinguish. This would make it possible to develop better 
ways to deal with anxiety as well as with more positive urges. 


A Final Word 

It is important to stress the nature of our attempt to deal with per- 
sonality and adjustment in terms of the concepts of conditioning. “We do 
not feel that the psychology of learning, as it now exists, contains all, or 
even very many, of the answers to the questions which arise in the study 
of personality. In fact we cannot be sure that the psychology of learn- 
ing can ever supply these answers. On the other hand, it is at least possible 
that such topics as repression, the unconscious, and psychotherapy will be 
understandable in terms of the processes of conditioning. Explorations of 
this possibility appear to be worth an occasional try. 



GLOSSARY 


Prominent in the glossary are terms specific to conditioning, inclading 
translations of Pavlov’s terms, which are designated by his name. The 
index may serve in locating the usage of words and expressions not in- 
cluded in the glossary. 

adaptation. The decrement in a response which is a consequence of its re- 
peated elicitation. (Cf. internal inhibition; contrast with extrinsic inhibition, 
interference) 

alpha conditioning (Hull). Augmentation by reinforcement of the original 
response to the conditioned stimulus. Not often observed. A more usual effect 
is adaptation of this response. (Cf. sensitization, pseudoconditioning) 

alpha response. Original response to the conditioned stimulus. (Cf. alpha 
conditioning) 

association. The establishment of functional relations between psychological 
activities and states in the course of individual experience. A broader term 
which includes conditioning. (Cf. conditioning) 

• associative shifting (Thorndike). Considered by Thorndike to be the more 
general case of which the conditioned response is a special ease. 

« avoidance training. A training procedure in which the learned movement cir- 
cumvents or prevents the appearance of a noxious stimulus. 

backward conditioning. The experimental arrangement in which the condi- 
tioned stimulus is presented after the cessation of the unconditioned stimulus. 

beta response (Grant). In eyelid conditioning, an unconditioned eyeblink to 
light sensitized by dark adaptation. 

» classical conditioning experiment. An experiment after the prototype of 
Pavlov, which consists in the repeated presentation of the conditioned and un- 
conditioned stimuli in a controlled relationship so that there occur alterations 
in reaction tendencies with respect to the conditioned stimulus which would not 
arise except for its relationship to the unconditioned stimulus and response. 
Distinguished from instrumental conditioning. 

concentration (Pavlov). I. The focussing of the hypothetical nervous pro- 
cesses of excitation and inhibition within restricted cortical areas. (Contrast 
with irradiation) 2. Behariorally, restricted generalization. (Cf. generalization) 
‘conditioned inhibition (Pavlov). A variety of internal inhibition defined by 
the experimental arrangements under which it is obtained. The negative, non- 
reinforced, combination to be differentiated from the positive conditioned 
stimulus consists of the positive stimulus pins an added stimulus known as the * 
conditioned inhibitor. 

conditioned response. A response which appears or is modified as a conse- 
quence of the occurrence of a conditioned stimulus in proximity to reinforce- 
477 
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ment. ( Svn. conditioned reflex or conditional reflex Pavlov; association reflex 
Bekhterev; individual reflex— Beritov) 

• conditioned stimulus (Pavlov). The experimental stimulus to vrhiek a nevr or 
modified response becomes related through the process of conditioning. (Syn. 
conditional stimulus, inadequate stimulus, substitute stimulus) 

* conditioning. The process of training which results in the formation of con- 
ditioned responses. (Cf. association) 

r contiguity theory. The hypothesis that, for learning to occur, it is necessary 
only for two stimuli (CS and UCS) or a stimulus and a response to occur 
closely together in time. (Contrast with effect theory and expectancy theory) 
continuity theory. Any theory of learning which incorporates the assumption 
that learning occurs gradually. (Contrast with non-continuity theory) 

counter conditioning. Extinction under circumstances in which the response 
decrement is hastened by the reinforcement of a response which displaces the 
original conditioned response. (Distinguish from adaptation and inhibition; ef. 
interference) 

delayed conditioning (Pavlov). The conditioned stimulus begins from 5 
seconds to several minutes before the unconditioned, and continues until re- 
inforcement occurs. (Distinguish from trace conditioning) 

differential inhibition (Pavlov). A form of internal inhibition arising in 
respect to the negative stimulus when stimuli are discriminated by the method 
of contrasts. (Cf. discrimination) 

v discrimination. The inferred basis for differential responses to unlike stimuli. 
Discrimination is always measured by differential responses, but absence of a 
specified differential response is not necessarily an indication of lack of dis- 
criminatory capacity. 

. disinhibition (Pavlov). A temporary increase in the strength of an extin- 
guished conditioned response as the result of an extraneous stimulus. While 
called by Pavlov an “inhibition of inhibition,” the term may be used in a strict- 
ly behavioral sense. 

dominance (Ukhtomsky). A physiological description of the interaction of 
certain reflexes, whereby following repeated stimulation, the dominant one is 
evoked by stimuli appropriate to other reflexes, while at the same time the 
normal responses to these stimuli are inhibited. 

effect, empirical law of. A statement of the fact that certain events placed 
appropriately with respect to a response leads to the strengthening of that 
response. (Contrast with effect, theoretical law of) 

effect, theoretical law of. The hypothesis that some satisfying or rewarding 
consequence is necessary for learning. (Contrast with effect, empirical law of; 
cf. effect theory) 

effect theory. Any theory of learning which stresses the theoretical law of 
effect. 

elicited behavior (Skinner). A response to a specifiable stimulus (Contrast 
with emitted behavior, operant; cf. involuntary behavior) 

emitted behavior (Skinner). Spontaneous behavior in response to no speci- 
fiable stimulus. (Contrast with elicited behavior, respondent; cf. voluntarv 
behavior) 
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error factor theory (Harlow). A theory designed to account for the phe- 
nomena of discrimination learning, particularly for learning sets (q.v.). The 
basic assumptions are (1) that the correct response is already in the response 
repertoire of the organism, but (2) that it fails to appear at once because of 
errors of certain specified kinds. Discrimination learning consists in the extinc- 
tion of these tendencies. Learning sets develop because of the transfer of such 
extinction to similar problems. 

* escape training. An instrumental conditioning proceedure in which the re- 
sponse terminates a noxious stimulus. (Contrast with avoidance training) 
excitation (Pavlov). One of two hypothetical nervous processes in the cortex, 
interacting with inhibition, its opposite. (Cf. inhibition) 

expectancy theory. The theory which interprets the animal’s learned behavior 
as consonant with anticipated (expected) events or consequences. (Syn. sign 
learning — Tolman; distinguish from effect theory, contiguity theory) 
experimental extinction. See extinction. 

experimental neurosis (Pavlov). A disturbed state appearing in some animals 
as a result of conditioning procedures, such as too difficult discriminations or 
prolonged periods of delay. The animal is unsuitable for experimentation, but 
may or may not appear to be disturbed when outside the experimental situation. 

external inhibition (Pavlov). A temporary reduction in the strength of a 
conditioned response as a result of the occurrence of an extraneous stimulus. 
(Contrast with disinhibition) 

extinction (Pavlov). The specific procedure of presenting the conditioned 
stimulus unaccompanied by the usual reinforcement; also the decrement in a 
conditioned response which results from that procedure. 

extinctive inhibition (Pavlov). The variety of internal inhibition which ap- 
pears with non -reinforcement of a conditioned response. (Cf. extinction) 

facilitation. A temporary nonlearned increase in strength of response as a 
result of the occurrence of a second stimulus or response. (Distinguish from 
reinforcement) 

• fixed-interval schedule. A schedule in which reinforcement occurs for the first 
response after a certain amount of time following the preceding reinforcement. 
(Contrast with variable-interval, variable-ratio, and fixed -ratio schedules) 
v fixed-ratio schedule. A schedule in which reinforcement occurs regularly on 
every n" trial. (Contrast with fixed-interval, variable-interval, and variable- 
ratio schedules) 

fractional anticipatory (or antedating) goal response. A component of the 
consummatory or goal response which can occur in the absence of the goal 
stimulus or reinforcer. Often used as a theoretical concept by S-R theorists to 
explain expectant behavior. Symbolized r G . 

goal gradient (Hull). (1) The delay-of-reinforeement gradient. (2) A per- 
formance gradient in which parts of a serial act are reinforced by a common 
goal situation, acts being more vigorous the nearer they He to the reinforcing 
state of affairs. The goal gradient so defined is explained in part by the gradient 
of reinforcement, but is a gradient of strength of response rather than of 
strength of conditioning and is modified by factors other than proximity of 
individna] responses to the reinforcement. 

habit. A term used formally (null) or informally to refer to the result pro« 
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dueed in the organism by training. Sometimes employed as a synonym for 

learning. . ... 

habit-family hierarchy (Hull). A number of habitual alternative behavior 

sequences having in common the initial stimulus situation and the final reinforc- 
ing state of affairs, the alternative sequences having a preferential order which 
produces the hierarchy. 

higher-order conditioned response (Pavlov). A conditioned response based 
on reinforcement by another conditioned stimulus and response. A first-order 
response is re inf orced by an unconditioned stimulus, a second-order response 
(or secondary response) by a first-order conditioned stimulus, and so on. (Cf. 
secondary reinforcement) 

hypothetico-deductive method. Method of theory construction in which a 
known (for example, mathematical) system is co-ordinated to a less well-known 
empirical one, in the hope that deductions in the known system will yield testable 
theorems in the more poorly understood one. Hull’s use of this method was not 
truly hypothetico-deductive, in that the formal process of co-ordination was 
not employed. 

inhibition. 1. (Pavlov). One of the two hypothetical cortical processes, inter- 
acting with excitation, its opposite. 2. Behaviorally, any decrease in the strength 
of a response which is occasioned by positive stimulation, either of the response 
itself (as in extinction) or of some other response (as in external inhibition). 
(Cf. conditioned inhibition, external inhibition, extinctive inhibition, inhibition 
of delay, inhibition of reinforcement, internal inhibition) 

inhibition of delay (Pavlov). That form of internal inhibition manifested 
during the latent period of a delayed or trace conditioned response. That the 
interval of delay is inhibitory is evidenced by (1) the ease with which disin- 
hibition may be demonstrated, and (2) secondary inhibition of weak conditioned 
responses if the attempt is made to elicit them during the period of delay 
(Rodnick). 

inhibition of inhibition. (Cf. disinhibition) 

inhibition of reinforcement (Hovland). The diminution of a conditioned re- 
sponse which may occur within reinforcement trials if such trials are repeated 
at short intervals. The phenomenon is like adaptation in that it is temporary 
and may be recovered from by rest. 

inhibition with reinforcement (Pavlov). (Cf. inhibition of reinforcement) 
inhibition theory. A theory of extinction which relies heavily upon a concept 
of extinctive inhibition. 

interference. The decrement in a response which is a consequence of the 
elicitation of another incompatible response. 

interference theory. A theory of extinction which relies heavily upon the 
concept of interference. 

. intermittent reinforcement. A training procedure in which the reinforcement 
occurs on only a fraction of the trials. Under some circumstances random 
alternation of reinforcement and non-reinforcement leads to a normal acquisi- 
tion rate and, at the same time, increases resistance to extinction. Special forms 
of intermittent reinforcement have different consequences. (Cf. fixed-ratio 
schedule, variable-ratio schedule, fixed-interval schedule, variable-interval 
schedule) 



GLOSSARY 


481 


internal inhibition (Pavlov). That form of inhibition which depends upon 
stimulation by a conditioned stimulus. (Contrast with external inhibition) 
interoceptive conditioning (Kazran). Conditioning in which either stimuli 
or responses are internal. 

investigatory reflex (Pavlov). (Cf. pure-stimulus act) 
irradiation (Pavlov). 1. The wave-like spread of excitation and inhibition 
through the cortex, the inferred physiological basis of generalization. (Contrast 
with concentration). 2. Behaviorally, the phenomenon of stimulus generaliza- 
tion. Little used in this latter sense. 

latent extinction. Extinction without responding which occurs as a result of 
non-reinforced experience with the goal box in a maze. Also called indirect 
extinction, goal extinction, and non-response extinction. (Cf. latent learning) 
latent learning. 1. Learning not made manifest in performance. 2. Learning 
which is latent in sense 1 (above) but, in addition, has occurred without rein- 
forcement, appearing only with the introduction of reward. 

learning. A relatively permanent change in response potentiality which 
occurs as a result of reinforced practice. 

learning set. The improved ability to master certain kinds of learning prob- 
lems which occurs as a result of previous experience with problems of the same 
sort. Has been demonstrated most often in experiments on discrimination learn- 
ing, but has also been demonstrated in other learning situations. 

micromolar theory (Hull, Logan). Any theory of learning which makes the 
primary assumption that responses with different quantitative values must he 
treated as different responses. 

method of simultaneous presentation of stimuli. A procedure for the study 
of discrimination learning. Positive and negative stimuli are presented together, 
and the organism must select one or the other. Sometimes called the method 
of choice reaction, 

method of successive presentation of stimuli. A procedure for the study of 
discrimination learning. Positive and negative stimuli are presented one at a 
time. Responses to the positive stimulus are reinforced; responses to the nega- 
tive stimulus are not reinforced. Pavlov called this the method of contrasts. 

mosaic of functions (Pavlov). A conception of localization of functions in 
the cortex. For Pavlov, the mosaic is determined in part anatomically (the 
analyzers), in part functionally, as a result of the integration of conditioned 
excitation and inhibition. 

negative conditioned stimulus. A conditioned stimulus which, through re- 
peated presentation, either alone or in combination, without being followed by 
reinforcement, tends to be followed by reduced or inhibited conditioned re- 
sponses. 

negative induction (Pavlov). The intensification of inhibition under the 
influence of preceding excitation. (Contrast with positive induction) 
non-continuity theory. Any theory of learning which assumes that learning 
occurs in a single exposure of the organism to the situation. (Contrast with 
continuity theory) 

observing response. (Cf. pure-stimulus act) 

omission training. A little used training procedure in which positive rein- 
forcement occurs unless the organism makes a specified response. 
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c Operant (Skinner). An act which first appears as a random movement. The 
operant commonly becomes related to a discriminated stimulus, which then sets 
the occasion for the operant behavior, but, strictly speaking, does not elicit it. 
(Cf. emitted behavior; contrast with respondent, elicited behavior) 
orienting reflex. (Cf. pure-stimulus act) 

paradoxical phase (Pavlov). A state of the cortical processes lying between 
that of equalization and complete inhibition in which only weak conditioned 
stimuli lead to responses and strong stimuli either elicit no responses at all or 
have a barely noticeable effect. (Cf. phase of equalization; ultra-paradoxical 
phase) 

- partial reinforcement. See intermittent reinforcement. 

phase of equalization (Pavlov). A state of the cortical processes lying be- 
tween that of normal waking and sleeping in which all conditioned stimuli, 
independent of their intensities, are equally effective. (Cf. paradoxical phase; 
ullra-paradoxical phase) 

positive induction (Pavlov). The intensification of excitation under the in- 
fluence of preceding inhibition. (Contrast with negative induction) 

preparatory response. A form of conditioned response within classical con- 
ditioning, often unlike the unconditioned response, which may be appropriately 
characterized as a response in readiness for the appearance of the unconditioned 
stimulus. 

presolution period. In discrimination learning, the period during which the 
animal responds in a chance fashion. A question of great theoretical interest 
is whether learning of the correct habit occurs during this period. (See con- 
tinuity and non-continuity theory) 

primary reinforcement (reward). A stimulus with innate reinforcing value 
which does not depend upon the fact of prior association with other reinforcing 
stimuli. (Cf. secondary reinforcement) 

pseudoconditioning (Grether). The strengthening of a response to a pre- 
viously neutral stimulus through the repeated elicitation of the response by 
another stimulus without paired presentation of the two stimuli. (Cf. do min ance; 
sensitization) 

punishment training. An instrumental training procedure in which a specified 
response is followed by the presentation of a noxious stimulus. 

pure-stimulus act (Hull). An act which serves no biological function other 
than to provide stimuli which aid in the integration of the organism’s behavior. 

recency principle (Guthrie). An explanatory principle of associative learning 
which states that an organism when next confronted with a stimulating situa- 
tion closely resembling an earlier one tends to react as it last did in such a 
situation. To he distinguished from that form of the “law of recency” which 
states that associative strength is a function of the time elapsed since the 
association was established. Guthrie’s principle is that of recency in order of 
occurrence, not of recency in time. 

receptor-exposure act. (Cf. pure-stimulus act) 

reconditioning. The re-establishment of a conditioned response by reinforce- 
ment after it has been diminished by extinction or forgetting. 

redintegration. The arousal of a response by a fraction of the stimuli whose 
combination originally aroused it. 



GLOSSARY 


483 


t reinforcement (Pavlov). 1. The experimental arrangement of presenting the 
conditioned stimulus accompanied by the unconditioned stimulus, or, more gen- 
erally, the arrangement of following the conditioned stimulus and response by 
reward or punishment, or some substitute for these. 2. The process which in- 
creases the strength of conditioning as a consequence of these arrangements. 
(Distinguish from facilitation; cf. primary reinforcement, secondary reinforce- 
ment, intermittent reinforcement) 

* respondent (Skinner). That form of response which is elicited by a stimulus, 
and hence to be distinguished from an operant. (Contrast with operant) 
response. A stimulus-occasioned act. A general term for acts correlated with 
stimuli, whether the correlation is untrained or the result of training. 

response differentiation. The modification of the quantitative characteristics 
of a response by reinforcing only limited forms of it. 

response generalization. The behavioral fact that following the conditioning 
of one response, other responses may he elicited by the same conditioned 
stimulus although there has been no specific training to establish a relationship 
between this stimulus and these responses. (Cf. stimulus generalization) 

response tendency. A tendency to respond which is inferred from other 
responses. The tendency may be expressed in units of behavior if it is inferred 
from measured behavior. 

reward training. An instrumental conditioning procedure in which a specified 
response is followed by a positive reinforcer. 
r G . (Cf. fractional anticipatory goal response) 

secondary extinction (Pavlov). The extinction of other conditioned responses 
which results when any one conditioned response undergoes extinction. (Syn. 
generalized extinction; cf. generalization) 

secondary reinforcement (reward). A stimulus which derives its reinforcing 
value from prior conditioning in which it has been associated with a primary 
reward. 

sensitization (Wendt). The increase in the strength of a reflex originally 
evoked by a conditioned stimulus through its conjunction with an unconditioned 
stimulus and response. Differs from conventional conditioning in that the re- 
sponse which is strengthened is appropriate to the conditioned stimulus, not to 
the unconditioned stimulus. (Syn. alpha conditioning — Hull; cf. psendocondi- 
tioning — Grether) 

sensory conditioning. (Cf. sensory preconditioning) 

sensory preconditioning. Two stimuli, Si and S 2 , are presented closely to- 
gether in time for a number of trials. Then a response is conditioned to one 
of these two stimuli. Finally the other stimulus is presented to determine whether 
the CR transfers to it. Sensory preconditioning is used to refer (a) to the 
experimental procedure and (b) to the fact of transfer of the CR from one 
stimulus to the other. (Cf. latent learning) 

sign learning (Tolroan). The formation of an expectation that a first stimulus- 
object is going to he followed by a second stimulus-object, as testified 1 ) 3 * be- 
havior preparatory to the second stimulus-object. (Syn. S-S learning, cognitive 
learning) 

„ spontaneous recovery (Pavlov). The return in strength of a conditioned 
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response, whether partial or complete, brought about by lapse of time following 
its diminution by extinction. 

stimulus. An external or internal object or event which occasions an alteration 
in the behavior of the organism. 

stimulus generalization (Pavlov). The behavioral fact that a conditioned 
response formed to one stimulus may also be elicited by other stimuli which 
have not been used in the course of conditioning. If one stimulus is like another 
except for a difference in a single dimension (for example, pitch or loudness 
of tones as determined by psychophysical experiments) the degree of generaliza- 
tion will vary inversely with the distance of the stimuli from each other along 
this dimension. The function expressing this relationship is called the generaliza- 
tion gradient. Its form varies with circumstances of experimentation. There is a 
corresponding gradient of generalization of extinction resulting when each of 
several similar stimuli evokes a conditioned response, and then the response to 
one of them is extinguished. Stimuli otherwise unlike may participate in gener- 
alization through the equivalence of the responses which they evoke. Such 
generalization is known as mediated generalization. (Cf. irradiation; response 
generalization) 

strength o£ conditioning. The extent to which a conditioned stimulus tends 
to evoke a given conditioned response, as inferred from the strength of response 
to the conditioned stimulus. Strength of response is not, however, a direct 
measure of strength of conditioning, for in inferring strength of conditioning, 
due regard must be paid to prior and subsequent responses. Thus, of two 
responses equal in magnitude, one may represent an inhibited response, another 
may be facilitated, and strength of conditioning may differ accordingly. Simi- 
larly, responses unequal in magnitude may represent equal strengths of condi- 
tioning if the conditioned stimuli differ in intensity. (Syn. associative strength; 
distinguish from strength of response) 

strength of response. Any descriptive characteristic of single responses (such 
as latency or amplitude) or of groups of responses (such as percentage fre- 
quency or rate) which may be expressed quantitatively and hence enter as data 
with respect to magnitude or vigor of response. Which measures of strength of 
response are most serviceable must be determined empirically. (Distinguish 
from strength of conditioning) 

subgoal. A stimulus regularly reacted to in the series of events leading up 
to the goal, so that this stimulus derives reinforcing value from its association 
with the goal. It is, therefore, a secondary reinforcer (q.v.). 

summation. The additive effect observed when stimuli which have been 
separately conditioned to the same response are presented together. 

temporal conditioned response. When an unconditioned stimulus is presented 
at regular intervals, a conditioned response may appear at or near the usual 
time of appearance of the unconditioned stimulus, even if the unconditioned 
stimulus is omitted. Such response is known as a temporal conditioned response. 

test trial. A trial vrith the usual reinforcer omitted introduced into a series 
of reinforced trials to determine whether a response occurs. 

top capability of the cortical cells (Pavlov, Konorski). The theoretical upper 
limit of the excitatory response of the cortical elements. 

trace conditioning (Pavlov). The conditioned stimulus is permitted to act 
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and tlien is removed before the onset of the unconditioned stimulus. Trace con- 
ditioned responses are more difficult to establish than delayed conditioned re- 
sponses. In short-trace conditioning the conditioned stimulus is terminated a 
few seconds before the unconditioned stimulus is presented; in long-trace con- 
ditioning the conditioned stimulus is terminated a minute or more before the 
unconditioned stimulus begins to act. (Distinguish from delayed conditioning) 

token. A secondary reward characterized by the fact that it must be manipu- 
lated in a prescribed manner in order to produce the primary reward. (Cf. 
secondary reinforcement) 

i trial-and-error learning (Thorndike). The mode of learning in which the learner 
tries various movements in its repertory, apparently in a somewhat random 
manner and without explicit recognition of the connection between the move- 
ment and the resolution of the problem situation. Tentative movements which 
succeed are more frequently repeated in subsequent trials, and those which fail 
gradually disappear. (Syn. selecting and connecting — Thorndike; contrast with 
associative shifting — Thorndike) 

types of nervous system (Pavlov). A classification of individual differences 
in conditioning on the basis of the relative susceptibility to excitation and inhi- 
bition. The final types, with their correspondence to the ancient Greek classifica- 
tions of temperaments, are as follows: (1) excitable type (choleric), (2) central 
type, with two subtypes, lively type (sanguine), and stolid type (phlegmatic), 
(3) inhibitable type (melancholic). 

ultra-paradoxical phase (Pavlov). A transitory state of the cortical processes 
sometimes occurring before complete inhibition, in which only inhibitory agents 
have a positive effect. (Cf. phase of equalization; paradoxical phase) 

« unconditioned stimulus (Pavlov). That experimental stimulus in a classical 
conditioning experiment which evokes the unconditioned response and hence 
serves as the reinforcing agent. (Syn. unconditional stimulus; conditioning 
stimulus — Razran; original stimulus — Guthrie) 

. variable-ratio schedule. A schedule in which reinforcement occurs irregularly, 
but on the average, after n responses. (Contrast with variable interval, fixed- 
interval, and fixed-ratio schedules) 

variable-interval schedule. An irregular schedule in which reinforcement 
occurs on the average after a certain amount of time. (Contrast with fixed- 
interval, fixed-ratio, and variable-ratio schedules) 

vicarious trial and error. The tentative behavior of looking hack and forth 
often observed in the behavior of rats at the choice points of mazes. 

Vincent curve. A technique for constructing learning curves in which per- 
formance measures for each subject are computed for constant fractions of the 
total number of trials required to reach a criterion of learning. This method 
permits the construction of average learning curves for groups of subjects which 
differ in number of trials to learn. 
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logical theory, 32-37; Skinner’s view 
of, 29 

Place learning, 223-224 
Plateau, 127 

Pleasure principle (Trend), 438 
Post-discrimination gradient, 345-347, 
3G5-366 

Practice, problem of the nature of, 6 
Predicate thinking (Treud), 439 
Prepotent response theory of reinforce- 
ment, 293, 265 

Presolution period in discrimination 
learning, 132-133, 482 
Primary process (Freud), 438 
Psendoconditioning, 60-64, 482 ; in studies 
of backward conditioning, 158 
Psychoanalysis, 437-440 and 436-477 
passim 

Psychotherapy, 467-473 
Punishment training, 71, 482 
Pupillary conditioning, 51, 51n. 

Pure stimulus act, 233 


Quality of reinforcement, 138-139 
Quantification of reaction potential 
(Hull), 113 


Kate of responding as a measure of re- 
sponse strength, 68-69, 80 
Rate-reinforcement schedules, 104 
Reaction potential (IIull), 129-131 
Reality principle (Freud), 439 
Reflex. See Conditioned reflex, Orienta- 
tion reflex, Unconditioned reflex 
Reflex action of tho brain (Scchenov), 20 
Reflex reserve (Skinner), 5n. 
Reinforcement, 5, 10, 06, 137-280, 483. 
Sec more specific topics: Delay of 
reinforcement, Iiaw of effect. Secondary 
reinforcement, etc. 

Reinforcement, secondary. See Secondary 
reinforcement 

Reliability of measures of conditioning, 
110-111 , 
Reminiscence, 125; of gcncraliied re- 
sponds, 319 

Repetition compulsion (Freud), 430 
Repression, 401-462 

Resistance to extinction ** s measure of 
conditioning, 113 


Respondeat behavior (Skinner), 73, 483 
Response, 483. See more specific topics: 
Response generalization, Response unit 
hypothesis, etc. 

Response compatibility, in relation to 
effect of motives on performance, 413- 
416 

Response competition, 142-144; impor- 
tance in studies of motivation, 405-40S. 
See also Conflict 
Response differentiation, 91-92 
Response generalization, 88-91, 358 
Response measure, importance in studies 
of motivation, 400-402 
Response, problem of specification, 109 
Response-unit hypothesis, 291-292 
Retention of conditioned responses, 281 
Reward training, 67-68, 483 
r 0 . See Fractional anticipatory goal re- 
sponse 

r 0 -> s 0 . .See Fractional anticipatory goal 
response 

Russian studies of conditioning, £0-22 

Schedules of reinforcement, 160-164; and 
secondary reinforcement, 188-192 
Secondary extinction, 483 
Secondary motivation, 424-433 
Secondary process (Freud), 439 
Secondary reinforcement, 97-99, 167-202, 
483; and delay of reinforcement, 151- 
153; and discrimination learning, 192- 
194; and the discrimination hypothesis, 
310-320; and the mediation of gap* 
time, 196-200; as a contributor to per- 
formance under schedules of partial 
reinforcement, J 01-102; as a function 
of the amount of primary reinforce- 
ment, 185-186; as a function of drive, 
388-191 ; ns a function of interatimulus 
interval, 182-184; as a function of 
number of primary reinforcements, 186- 
183; based od negative primary rein- 
forcement, 173-176; demonstration ex- 
periment, IC7-IC9; developed through 
methods of classical conditioning, 169 
170; extinetion of, 180-182; generalisa- 
tion sero«s drives, IPG ; nature of, If^‘ 
170; provided l>r discriminative stim- 
uli, 171-173; provided by proprioceptive 
stimuli, 152- relation to Incentive 
motivation, 170 173; relation to rein- 
forcing vnlop of sweet ta»tr», 251-253 
Second signalling system (Pavlov), 3*5 
Relortive and nonwlrctlvs theories of drive 
stlmsLttien, 418-422 
Semantic general ltat ion, 351 353 
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Sensitization, 64-65, 483 
Sensory control of the maze, 224 
Sensory preconditioning, 215-221, 483- 
484; and r G -> s 0 mechanism, 216-221; 
and the interstimulus interval, 218-219 ; 
demonstrations of, 215; mediational in- 
terpretation, 216-221 
Sham feeding, 20, 243 
Sign-gestalt. See Expectancy theory 
Sign learning (Tolman), 484 
Social techniques (Tolman), 431 
Solution period in discrimination learn- 
ing, 132-133 

Speed-of-locomotion gradient, 149-150 
Spontaneous recovery, 82-83, 284, 483 
Spontaneous regression, 297-299, 307 
S-S theory. See Expectancy theory 
Statistical learning theory, 128-12*9 
Stimulational theory of nature of rein- 
forcement, 241-242, 249-263 
Stimulus generalization, 32, 36, 87-89 
148, 328-360, 484; absolute vs. relative, 
335-336; and motivation, 340-341; and 
number of reinforcements, 335-336; 
distinguished from response generaliza- 
tion, 328 ; effect of changing more than 
one aspect of the stimulus, 345; effect 
of discrimination training, 345-347 ; 
effect of stimulus intensity, 350-354 ; 
individual differences in, 3*39-340 ; in- 
fluence of partial reinforcement, 340; 
relation to perception of differences’ 
3a0-354; methods of investigation, 331- 
332; typical experiments, 329-330 
Stimulus intensity dynamism (Hum 14s 
342-344. See G?so Intensity of condh 
tinned stimulus, Effect on learning 
Stimulus substitution, as a theory of con- 
ditioning and learning, 23 
Stimulus variability and rate of condi- 
tioning, 129-131 
Structuralism, 16-17 18 

Subgoal, 484. See Secondary reinforce- 
ment 

Substitute behavior, 457-458 
Summation, 484 ; of conditioned reflexes 
bi, of generalized responses, 348-350 ’ 
Superego (Freud), 439 
Superstitious behavior, 197-198 
Symptom formation, 464-467 


Tandem schedules of reinforcement 
Tastes as reinforcement, 248 949 953 
Taylor scale. See Manifest Anxiety S( 
Temporal conditioning procedure 48 
Tension-reduction theory of nature of 
inforcement, 23S-240; and avoida 


learning, 266-277 ; drive, need, and 
drive-stimulus reduction as examples, 
239-240 

Thinking, 22, 461-462 
Threshold (Hull), 129-130 
Token rewards (incentives), 97 
Top capability of the cortical cells (Pav- 
lov, Konorski), 34, 484 
Trace conditioning procedure, 47, 484; 
generalization of responses based on, 
357-358 

Transformations, mathematical, of meas- 
ures of conditioning, 112 
Transituational nature of reinforcers, 207 
Transposition, 378-384; as a function of 
similarity of test and training stimuli, 
379-381; mediational interpretation, 
3S3-3S4 ; Spence’s theory, 379-381 
Trial-and-error learning, 67, 484; heter- 
ogeneous compound, 146-148 ; homoge- 
neous compound, 146-148 
Two-process (factor) theory, 99, 266-277; 
possible relation to learning with posi- 
tive reinforcement, 266n. 


UCR. See Unconditioned reflex 

UCS. See Unconditioned stimulus 
Ultraparadoxical phase (Pavlov), 34, 485 
Urn weg (detour) problem, 406-40S 
Unconditioned reflex, attempts to condi- 
tion with peripheral elicitation, 7-8; 
response to CS, 302-304 

Unconditioned reflex (response), com- 
pared with conditioned reflex, 52-53; 
defined, 46; reactions employed, 50-51 
Unconditioned stimulus, defined, 46, 485; 
on conditioning of its duration, 
4/3-274; onset vs. cessation in the re- 
inforcement of fear responses, 270-272 
Unconscious, 461-462 


Tacuum activity (Ethology), 425 
ana e-interval schedule of reinforce- 
ment, 162-164 

^ se ' le ^ u ^ e reinforcement, 


■'•i.uiciu, iou-'au 

v erbal conditioning, 102n. 

1 icarious trial and error, 233 485 
' lucent curves, 114-115 

°i09^ ar ^ con ^ r °l of involuntary behavior, 


^ciors m conditioning, 104 
oluntary response, as a source of error 
conditioning experiments, 5S-59; 
compared with reflex response, 56; con- 
ditionability of, 100-101 



